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FOREWORD 


The  Sixth  Joint  Military  -  Industry  Quided  Missile  Reliability 
5y?«posium,  sponsored  by  the  (Department  of  Defense,  was  conducted  by 
the iDepartment  of  Army  under  the  supervision  of  the  U»  S,l jAny  Ordnance 
Missile  Command. 

Due  to  the  length  of  the  material  presented,  it  was  necessary  to 
publish  the  technical  papers  presented  in  three  volumes.  The  first 
two  volumes  contain  the  unclassified  papers  and  Volume  3  the  classified 
ones. 

To  prevent  delays  in  publishing,  discussions  following  each  paper 
are  omitted  from  these  proceedings. 

Additional  copies  of  these  proceedings  can  be  obtained  by  request 
directed  to  Commanding  Oeneral,  Army  Rocket  and  Oulded  Missile  Agency, 
Redstone  Arsenal,  Alabama,  ATTN:  Technical  Library. 
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WELCOMING  ADDRESS 
Byi  Major  General  San  C.  Russell 
Com  art  ding  General,  D.  S.  Army  Air  Defense  Center 
Fort  Bliss,  Texas 


It  is  a  pleasure  to  welcome  you  to  the  Amy  Air  Defense  Center* 

This  area  is  a  particularly  appropriate  location  for  this  symposium 
because  of  our  extensive  activities  in  the  missile  field*  Including 
the  initial  Ajax  troop  firing  in  1953>  over  u,700  missiles  have  been 
fired  under  Fort  Bliss  control  by  using  troops.  In  addition,  there 
have  been  some  8,500  research  and  development  firings  conducted  by 
our  good  friends  and  neighbors  at  White  Sands* 

During  this  period,  the  annual  percentage  of  successful  Ajax 
rounds  in  troop  firings  has  increased  from  to  8h%,  and  Hercules, 
after  a  rather  unfortunate  start,  averaged  T5t  for  calendar  year  19?9* 

While  certainly  some  of  this  improvement  in  the  percentage  of 
successful  missiles  can  be  attributed  to  improved  training  and  im¬ 
proved  procedures,  a  great  deal  must  be  credited  to  the  Improvement 
in  the  reliability  of  the  systems  and  their  components.  For  example, 
there  have  been  almost  UOO  modifications  to  the  Ajax  system,  resulting 
largely  from  experience  gained  in  live  firings* 

Thera  has  been  some  discussion  of  eliminating  the  annual  service 
practice  of  surface-to-air  missile  units,  primarily  because  of  tbe 
cost  involved.  In  ny  opinion,  this  would  be  the  falsest  kind  of 
economy.  Without  going  into  the  training,  morale,  troop  confidence,  and 
public  relatione  benefits  gained  from  live  firings,  and  they  are  many, 

I  believe  the  firings  have  justified  themselves  from  the  product 
improvement  aspect  alone, 

Oentlemen,  1  wish  you  a  most  successful  symposium.  The  complete 
resources  of  this  center  are  available  to  support  this  very  worth¬ 
while  get-together. 


EBTK7TE  ADDRESS 


BIi  Dp.  I.  0.  Witting 
Deputy  Director  of  Research  and  Development 
Department  of  the  A rmy 


General  Rue sell,  Mr.  DeWitt,  n embers  of  the  Symposium,  it  la 
indeed  a  pleasure  and  honor  for  me  to  deliver  this  keynote  address 
at  the  6th  Joint  Military  Industry  Guided  Missile  Reliability 
Symposium.  As  you  know  from  your  programs,  I  am  filling  the  ahoes 
of  Mr.  Morse,  our  Director  of  Research  and  Development  for  De¬ 
partment  of  the  Army.  Unfortunately,  he  is  a  casualty  of  the 
budget  battle  in  Congress.  Mr.  Morse  has  asked  me  to  wxprese  his 
regret  in  being  unable  to  be  here  with  you.  Ton  see  me,  too.  In 
tbe  Any  have  our  reliability  problems  in  getting  our  speakers  to 
fulfill  their  commitments. 

Before  getting  Into  my  subject,  I  want  to  express  appreciation 
for  the  Department  of  Defense  to  General  Russell  and  members  of  the 
Fort  Bliss  Center  for  permission  to  uae  these  facllltiea,  and  to  the 
members  of  the  Army  Ordnance  Missile  Coms&nd  who  have  contributed 
eo  much  to  arranging  the  symposium.  I  know  a  lot  of  work  has  gone 
Into  the  preparation  for  this  symposium,  and  I  feel  certain  you  will 
all  reap  benefits  from  their  labors. 

Reliability  id  a  word  which,  in  the  vocabulary  of  many,  has 
become  cornnon  place  and  trite.  Too  often  it  has  received  much  lip- 
eervice  and  little  action.  This  kind  of  thinking  must  be  radically 
changed.  I  can  assure  you  that  the  Department  of  Defense  is  aware 
of  this  and  is  doing  ite  utmost  to  insure  that  the  solution  ef 
reliability  problems  Is  increasingly  stressed  in  the  eervices. 

In  talking  here,  I  do  not  expect  to  tell  you  how  to  do  your  Job 
I  may  not  tell  you  many  things  you  do  not  already  know.  1  hope, 
however,  I  can  impart  to  you  the  conviction  that  your  problems  can 
and  must  be  solved.  The  cost  of  reliability  failures  has  already 
conservatively  been  estimated  in  the  billions  ©f  dollars.  These 
costs  are  rising  each  day. 

There  may  be  some  of  you  in  the  audience  who,  along  with  syself 
can  remember  the  good  old  days  whan  automobiles  were  considered 
rather  unreliable  instruments  of  locomotion.  I  can  remember  taking 
short  trips  over  the  muddy  roada.  In  those  days  we  used  to  carry 
extra  gasoline,  pick  ajee,  shovels,  and  certainly  no  one  would  be 


without  extra  rope  to  help  pull  stalled  vehicles  out  of  muddy  holes. 
The  expression,  "net  out  and  get  under"  often  applied  to  motorists 
Jokingly  may  have  been  coined  frivolously  but  not  without  basis. 
Reliability  of  automobiles  increased  with  experience. 

Op-dating  ourselves  a  few  years,  let's  look  at  the  beginnings 
of  air  travel.  In  the  early  days  of  airplanes,  each  pilot  who 
survived  an  air  crash  told  what  he  could  remember  concerning  the 
cause  of  the  accident.  This  was  the  only  method  available  for 
determining  what  vent  wrong.  When  the  Oermans  were  building  V-2's 
during  the  Second  World  War,  they,  too,  were  going  through  the  same 
experiences  of  learning  reliability  from  the  accidents  that  took 
place.  Today  in  missiles  we  have  coma  a  long  way  from  the  beginnings 
just  as  we  have  with  the  aircraft  of  yesteryear. 

As  an  illustration,  I  would  like  to  tell  you  of  my  first 
commercial  airplane  trip.  It  was  in  a  bi-plane.  At  each  stop  the 
pilot  and  copilot  got  out  of  the  aircraft  and  pulled  and  pushed  on 
every  strut.  While  refueling  they  checked  the  wings,  the  under¬ 
carriage,  the  ailerons,  the  elevators.  In  essence,  neither  they 
nor  I  considered  that  aircraft  very  reliable.  The  story  is  con¬ 
siderably  different  today.  Through  trial  and  tribulation,  we  have 
learned  reliability  in  aircraft. 

In  the  missile  business  today,  we  have  neither  the  time  nor 
can  we  pay  the  price  for  learning  reliability  through  such  trial 
and  tribulation  methods.  What  then  have  we  been  doing  and  what  more 
can  we  do  to  improve  reliability  in  missiles?  I  have  several  con¬ 
victions  which  I  should  like  to  discuss  with  you  today. 

First  of  all,  1  am  convinced  that  we  must  build  reliability 
into  our  components.  Some  missiles  have  been  flown  without  adequate 
returns.  Development  missiles  cost  so  much  money  that  we  cannot 
afford  to  fire  them  into  the  air  Just  to  see  if  the  components 
function  properly.  In  many  cases  where  failures  result,  the  exact 
cause  of  failure  is  not  known;  in  fact,  the  specific  component  that 
failed  is  unidentifiable.  The  only  Justification  of  first  flight 
tests  is  to  measure  the  system-  environmental  and  stress  conditions. 

In  the  past,  too  much  attention  has  been  given  flight  tests  to 
the  detriment  of  laboratory  testing.  1  realise  that  much  bench 
testing  and  functional  testing  has  been  done,  but  the  importance  of 
other  asoects  such  os  testing  to  failure  has  been  largely  neglected 
to  the  detriment  of  reliability.  An  important  part  of  any  reliability 
program  Is  to  insure  that  samples  of  development  hardware  be  produced 
or  other*  or  procured  for  test  to  failure  in  your  laboratories.  It  is 


‘  m.- 1  mj 


r  >  a/  *_n  v  Kn  ia.~  fc,"\  m  ^  4  A 


essential  that  this  be  done  so  that  no  unreliable  component  finds 
Its  way  into  the  missile  system.  We  must  know  accurately  the  con¬ 
ditions  of  enriro resent  into  which  the  particular  component  Is  to 
function.  Only  then  can  ve  insure  obtaining  an  adequate  margin  of 
safety. 

I  feel  strongly  that  we  should  push  the  state  of  the  art  and 

*  advanced  technology  in  the  making  of  all  components.  Uhen  these 
components  have  been  proven  reliable,  then,  and  only  then,  should 
they  be  put  Into  a  system.  By  way  of  Illustration,  this  is  the 
philosophy  that  was  used  in  the  development  of  HIKE  AJAX  system. 

*  Ho  major  scientific  breakthroughs  were  to  be  counted  upon  for  the 
successful  completion  of  the  designs.  Too  often,  systems  are  brought 
well  along  Into  the  development  cycle  without  adequately  paying 
attention  to  these  considerations.  Then  quick,  and  generally  costly, 
"fixes"  are  needed  to  mend  the  situation* 

Hjr  second  conviction  is  that  you  cannot  Inspect  quality  into  a 
product.  You  have  to  design  it  into  the  product  and  build  it  into 
the  product.  Inspection  may  remove  all  the  Items  that  ere  beyond 
a  particular  specification  limit.  However  the  uncertainties  of 
insp?  lion  end  '’ateri  oration  with  time  end  use  may  give  early  high 
failure  rates.  This  iru..  •  ir.a  that  to  achieve  reliability  dependence 
must  be  placed  strongly  or>  Irf.v  a  of  particular  critical  characteristics 
as  determined  by  raer-rwement .1  or  samples  of  the  product  so  as  to 
get  warning  of  decreasing  :-arfc  r  with  respect  te  the  rejection 
points.  I*;  insure  reliability,  v  j  must  start  with  the  raw  materials 
i*nd  check  the  product  all  through  its  processing. 

This  brings  me  to  my  third  co  viction  —  that  is,  once  adequate 
reliability  is  designed  into  a  sysr.?n,  quality  control  program a  must 
be  instituted  to  insure  maintenance  of  inherent  reliability  throughout 
development  and  production.  In  the  development  cycle,  the  emphasis 
on  reliability  must  be  a/ihor»d  to  throughout  the  testing  program. 

The  entire  test  program  ehnOd  be  closely  correlated  with  actual  service 
conditions.  Furthermore,  ir creased  enphasie  must  be  placed  en 
determining  the  cause  of  fa' lores, 

T  c»n  wial  Imagine  some  of  you  in  the  audience  thinldng  that 
>>ar«  1  puling  .  .  ue  r-ci  pilots  out  of  their  crashed  alrplanee 

*  s.gftl t  to  interr  '  -b  thee:.  Th.\o  Is  e*lj  too  true,  but  at  least 

toi.oy  w:  ha*.,  much  better  to  give  ue  much  better  information 

will  out  the  lose  of  human  ljvts  d  ,^ng  a  development  program. 

,  ;'tif  1  of  pushing  reliability  through  development  of 

atomic  17 stems,  axic^atically  I  am  placing  greater  responsibility 
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and  reliance  on  producers  of  these  systems.  Because  of  the  complexity 
of  the  systems  being  developed  today,  the  producer  must  assume 
increasing  responsibility  with  regard  to  testing  and  adjustment  of 
missiles  prior  to  firing.  An  important  factor  in  this  regard  ie 
lo  have  contract  arrangements  which  centralize  responsibility  in  an 
unmistakable  way.  The  same  organization  should,  preferably,  be 
responsible  for  both  the  design  and  first  production  of  new  complex 
devices  and  systems.  It  would  be  most  beneficial  If  that  same 
organization  should  be  responsible  for  production  through  evaluation 
and  early  service  use  for  incorporation  into  the  design  of  the 
changes  found  desirable  and  necessary  by  such  experience  both  in 
use  and  in  production.  With  a  design  in  performance  so  established, 
the  matter  of  multiple  sources  of  production  can  then  be  approached 
on  a  sound  basis. 

The  stress  which  has  been  put  upon  reliability  In  atomic  war¬ 
heads  is  impressive.  These  warheads  are  now  being  designed  to 
require  a  minimum  amount  of  monitoring  and  testing  both  in  storage 
and  prior  to  use.  We  should  stress  the  same  goal  for  our  missile 
systems  so  that  they  may  be  fired  like  artillery  rounds  with  a  much 
higher  reliability. 

The  final  conviction  which  I  will  discuss  today  is  that  the 
problem  of  reliability  is  one  about  which  personnel  at  all  levels 
and  through  all  phases  of  development,  production,  and  use  must  be 
motivated  to  emphasize  and  to  solve.  This  problem  of  motivating 
personnel  and  of  educating  them  in  the  importance  of  reliability  is 
the  same  for  the  armed  services  and  contractors  alike.  During  the 
second  World  War  in  r.any  of  the  aircraft  factories,  large  and 
colorful  signs  stressed  the  need  for  reliability  in  the  production 
line.  These  signs  read  to  the  effect  that  "Some  day  your  brother 
may  be  in  the  aircraft  you  are  building," 

Now  I  have  told  you  how  important  I  think  reliability  is.  1 
have  given  you  a  few  of  my  convictions  on  the  Subject,  Before 
concluding,  however,  I  should  like  to  bring  out  three  factors  which 
illustrate  not  only  the  problems  that  must  be  faced  tut  the  goals 
that  must  be  achieved.  These  three  factors  are  cost,  complexity, 
and  time. 
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How  much  dees  reliability  coot  us?  This  question  has  never  been, 
nor  will  it  ever  be,  adequately  answered.  We  know  that  our  systems 
are  becoming  more  arid  more  expensive  and,  hence,  the  cost  of 
failure  mounts  continually.  For  instance,  the' NINE  AJAX  air  defense 
system  now  has  a  per  missile  production  cost  of  approximately  $25,000 
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while  its  successor,  the  NIKE  HERCULES,  cost  is  $66,000  per  aiaslla 
round.  The  dollar  figure  for  the  production  round  cost  of  a  NIKE 
ZEUS  missile  will  be  about  an  order  of  magnitude  greater  than  the 
cost  of  NIKE  AJAX,  From  a  dollar  point  alone,  the  costs  of  un¬ 
reliability  are  alarming,  Not  only  is  there  the  cost  of  the  missile 
1-  r:-  during  an  abort,  there  is  the  cost  of  the  operation  and  main¬ 
tenance  of  ground  support  equipment  during  the  firing  and  during 
•  the  development  program  and  training  program  there  is  the  added  cost 

of  drone  targets  for  air  defense  missiles.  -- 

Another  measure  of  the  cost  of  unreliability  may  be  gained  by 
looking  at  our  satellite  programs.  Pew  figures  have  been  released 
to  show  the  overall  costs  of  programs  of  this  nature.  One  program, 
whose  cost  figures  were  released,  showed  that  the  total  program 
cost  tne  government  $111  million,  or  $10  million  per  attempted 
firing.  Because  of  several allures,  the  cost  per  successful  firing 
rose  to  $3b  million.  That's  one  heck  of  a  lot  of  money  in  any  man's 
book.  In  addition  to  the  dollar  cost  with  relation  to  satellite 
programs,  soon  a  man's  life  may  well  be  added  to  the  cost  of  an 
abort  during  MERCURY  shots. 

Missile  system  complexity  may  be  measured  by  numerous  parameters, 
I  will  use  only  one  to  give  you  an  illustration  of  how  our  missile 
systems  are  becoming  increasingly  complex.  Using  the  number  of 
engineering  drawings  as  a  parameter,  the  NIKE  AJAX  system  required 
100  thousand,  while  the  NIKE  HERCULES  system  requires  120  thousand. 
Yet,  today,  before  development  is  completed  on  the  NIKE  ZEUS  system, 
it  will  be  necessary  for  approximately  b*>0  thousand  engineering 
drawing"  to  be  made. 

Turning  now  to  time,  there  are  no  adequate  time  data  available 
in  unclassified  literature.  Even  if  there  were,  these  data  would 
vary  greatly  considering  such  factors  in  air  defense  work  as  the 
type  of  target,  altitude  and  course  of  target,  defense  missile 
capabilities,  use  of  decoys,  etc.  It  is  apparent,  however,  as  the 
missile  threat  increases,  seconds  of  time  will  become  increasingly 
important  and  could  determine  the  fate  of  a  city  or  nation.  It  is 
for  this  reason  that  reliability  goals  of  99  percent  are  necessary. 

In  conclusion,  the  attainment  of  reliability  in  combat  involves 
many  things  and  many  people.  We  must  be  sure  that  we  are  aware  of 
all  of  them  and  that,  proper  measures  are  taken  with  respect  to  them. 
Reliability  involves  the  development  and  design  engineer;  it  may  be 
affected  by  the  specification  writer,  by  the  purchasing  agent,  by 
the  contracting  officer,  by  the  inspector,  and  by  the  quality  control 
organization.  It  Is  affected  by  the  responsibilities  set  up  by  the 
contract.  Everyone  has  to  do  a  Job  right  to  get  reliability.  Ite 
achievement  calls  for  a  proper  attitude  on  the  part  of  everyone. 
Conversely,  the  fa;  )nru  of  nry  link  can  break  the  chain.  Reliability 
is  everybody '.-i  bo  ■incss. 


vi 


~3fc 


‘jGi3kZiS~d6&-. *±.^  - 


i  vy^.'iSSfi 


r 


,v\j  i ru  *r-4  rv  jr-y  »  *  w~*  w  *  ■ 


T^<7T19ATj  reliability  or  fisip  army  gpitbd  misciiz  systems 

Colon* 1  Keith  H.  Evbank  Hq,  U SCON ARC 


uentlement 

For  the  next  few  d eye  reliability  will  be  discussed  from  every 
possible  aspect.  Maybe  we  should  start  by  defining  reliability,  and  of 
oonrss  this  depends  on  who  you  are.  If  you  are  a  statistician  -  it 
asaas  pur*  numbers.  To  bom*  engineers,  It  nay  Mean  performance  Halts 
of  specific  Materials  or  derlees.  To  one  who  write*  dictionaries  it 
reane  *  suitable  or  fit  to  be  relied  on,"  "worthy  of  dependence  or 
reliance."  ^trustworthy-"  and  of  couree ,  More. 

I  am  a  soldier,  so  I  an  going  to  give  you.  In  broad  terms,  the 
concept  as  the  soldier  In  the  field  sees  it;  of  tactical  reliability. 
and  I’m  not  speaking  of  the  soldier  la  s  warn  barracks  or  s  theater. 

I  mean  the  young  private  living  In  cold,  rough  combat  conditions  with, 
survival  as  on*  of  his  prlas  objectives. 

Field  Army  missiles  can  be  divldsd  into  ■o  general  classes  -  Air 
Defense  and  Artillery  ground  support  or  SSM  -  and  as  you  can  Imagine, 

In  many  ways  reliability  has  a  different  meaning  to  the  Air  Defense 
Artilleryman  than  It  does  to  tbs  field  Artilleryman.  They  have  in 
common,  the  requirement  that  a  system  place  effective  fire  on  a  target 
at  the  proper  time. 

Tactical  Reliability  in  Air  Defense  i  „  .  .  -sterlsed  by  continuous 
preparedness  for  action  and  effective  respc  a  the  reaction  time 
available.  In  certain  instances,  the  requl../u  reaction  time  la  of  ths 
utmost  Importance  and  can  be  achieved  only  if  tactical  reliability 
closely  approaches  design  reliability.  A  brief  comment  on  the  present 
and  future  Field  Army  Air  Defense  Systems  will  illustrate  the  user's 
perspective. 

Forward  Area  Air  Defense  weapons,  minimum  reaction  time  con¬ 
siderations  assume  prime  importance.  This  type  system  must  be~*  capable 
of  almost  Instantaneous  reaction  under  the  worst  possible  environmental 
conditions.  An  obvious  corollary  is  the  requirement  for  *n  absolute 
minimum  of  maintenance. 

The  REDEY2,  a  man- transportable ,  air  ■’efen.je  guided  missile  system, 
la  being  developed  to  moot  these  criteria.  Design  reliability  muet 
be  attained  through  weapon  simplification  and  ruggedisation.  However, 
the  fact  that  an  individual  soldier,  upon  detecting  an  enemy  target, 
alms  and  fires  the  missile,  lntroduceo  overriding  factors  of  the  human 
alamanl  into  the  syotev.  A  design  reliability  approaching  100%  is 
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therefore  necessary  In  order  to  minimise  the  degrading  effects  of  the 
human  factors  involved  and  thus  maximize  tho  final  tactical  reliability 
of  the  system*  Obviously,  a  weapon  of  this  type  must  bare  an  extremely 
rapid  reaotion  time,  or  it  is  not  "tactically"  reliable* 

In  MAUIEt,  a  mobile,  division-support  air  defense  guided  missile 
system,  an  almost  opposite  approach  will  be  taken  to  meet  the  require- 
x«nts  of  the  forward  battle  area*  Almost  fnl~\  automation  will  be 
utilized  in  order  to  reduoe  the  human  factor*  Automation,  invariably 
Introduces  equipment  complexity  and  its  attendant  maintenance  problems* 
Experience  has  shown  that  user  personnel  have  difficulties  with  equip¬ 
ment  complexity  and  resultant  maintenance  requirements  under  stringent 
tactical  conditions  expected  in  the  forward  area*  It  ie  evident, 
therefore,  that  acceptable  tactical  reliability  in  the  MAUIZR  system 
must  result  from  maximum  component  reliability  and  unsophisticated 
maintenance  procedures. 

As  the  demands  upon  system  reaction  time  become  lees  severe,  we 
can  tolerate  e  transition  from  full  automation  to  the  semi-automatic 
design  exemplified  by  the  HAWK  system.  It  is  evident  though,  since 
operational  tanning  levels  and  functions  are  intentionally  at  a  minimum, 
that  the  reliability  of  unattended,  remotely-controlled  equipment  must 
continue  to  reflect  full  automation  standards.  Even  with  ee  mi -automatic 
design,  tbs  problems  of  maintaining  personnel  efficiency  are  outweighed 
by  the  need  for  reliable  equipment  performance  for  extended  periods  of 
readiness  under  conditions  when  little  ox  no  maintenance  may  be  per¬ 
formed.  Tactical  reliability  starts  with  axiaised  design  reliability 
which  can  be  supported  in  toe  field  by  quiet,  simple,  and  reliable 
maintenance  procures. 

Longer-Rag  a,  Air  Defense  systems  dc  not  requirv  the  extr®-  , 
fast  reaction  '  m.  f  their  low  a^-itu'’  ,  shorter  range  counter]  , 
nor  are  they  subjected  to  the  extremes  f  rough,  handling  and  environ¬ 
mental  stress.  The  emphasis  mist  be  placed  upon  achieving  and  main¬ 
taining  a  satisfactory  degree  of  tactical  r  .dines*.  With  the  generally 
larger  and  more  complex  equipment  associated  with  long  range,  air 
defense  systems,  the  ualntenance  problem  tends  to  become  acute.  The 
ratio  of  maintenance  time  to  operational  time  ie  a  critical  factor. 

The  required  t  tical  reliability  can  be  achieved  only  by  Incorporating 
a  design  relis-billt.  such  that  operation  for  extended  periods  can  be 
expected  with  little  maintenance  and  few  unacceptable  losses  in 
operating  time. 

Now  for  SoM  or  field  artillery  systems*  Although  we  have  the 
equally  important  requirements  for  effectiveness,  operator  training 
and  timeliness,  the  parameters  are  not  the  sane  as  Air  Defbnae  weapons 
because  of  different  priorities  of  characteristics. 

To  the  field  artilleryman  reliability  is  the  level  of  assurance 
that  the  system,  will  effectively  accomplish  the  assigned  mission. 


that  la,  plaoa  effectiee  fire  on  the  target  at  the  proper  ties.  This 
effectiveness  la  the  tactical  reliability  of  the  systea  and  the  tech¬ 
nical  reliability  of  the  coapooanta  of  the  aystea. 

By  tabtieal  reliability  we  naan  the  ability  of  the  system  -  under 
the  required  tactical  and  field  environmental  ooodltlona  -  (l)  to  place 
effective  fire  on  the  target,  and  (2)  to  oa -operated  by  ar tiller;' 
soldier*  with  average  training* 

Effective  fire  is  that  which  is  accurately  placed  on  the  desig¬ 
nated  target  at  tha  desired  tlaa  to  produos  ths  desired  effect*  It  la 
both  accurate  and  ties It. 

Leet  someone  accuse  n  of  confusing  accuracy  and  reliability, 
remember  that  to  ths  tactical  commander  it  nattars  little  whether  a 
mission  fails  because  of  an  unreliable  assembly  or  inaccurate  flight* 
The  fact  that  the  system  has  not  plaeed  accurate  and  timely  fire  on  tha 
target  has  demonstrated  Its  unreliability  for  that  particular  mission* 

In  addition,  the  system  must  be  capable  of  being  operated  by 
average  soldiers*  Remember  that  the  apparent  or  dealgn  reliability 
of  a  system  can  be  materially  increased  if  skilled  technicians, 
specialised  teat  and  maintenance  equipment  and  a  sufficient  level  of 
replaoeasnt  components  are  used  in  the  firing  area*  However,  our  war¬ 
time  turnover  due  to  casualties,  sickness,  accident,  and  rotation  la 
such  that  we  cannot  perform  HAD  type  firings  in  combat.  Alto,  con¬ 
sidering  the  mobility  expected  of  our  future  missile  systems,  we  cannot 
expect  lamed late  contractor  or  even  Ordnance  support.  Therefore,  when 
you  are  designing  a  tactically  reliable  system,  ask  yoursolf  these  two 
questlonat 

Do  the  prefiring  adjustment*  or  replacements  require  a  skill  level 
higher  than  that  posseseed  by  average  Artillery  soldiers! 

Do  the  prefirlng  adjustments  or  replacements  require  equipment  or 
spares  which  would  not  normally  be  employed  by  the  Artillery  battery? 

Technical  reliability  nay  be  considered  broadly  as  tbo  composite 
reliability  of  tbe  components  of  tbe  missile  syntem.  This  must  include 
all  ground  and  toot  equipment,  as  well  as  the  witisile  itself,  W  are 
interested  In  the  whole  sequence  of  operations,  rot  just  in  tie  in¬ 
flight  reliability  of  tbe  missile*  In  this  roop^ct,  oversophii«tl cated 
systems  are  particularly  to  be  avoided. 

Most  of  you  are  familiar  with  military  characteristics,  a  statement 
of  tbe  capabilities  of  r  missile  system:  vhleh  enable  it  to  ful  fill  the 
Qualitative  Materiel  Requirement, 


I  would  like  to  giro  bom  examples  of  bow  wo  In  Materiel  Dotb lop- 
wen  ta  Interpret  these  statement*  to  provide  you  with  a  better  feel  for 
the  level*  of  reliability  and  tin#  limit*  required  for  field  artillery 
guided  mlaaile  system*.  Suppose  we  require  -  and  thl*  1*  typical  -  of 
SSM  systems  -  that  90  percent  of  the  missiles  removed  from  six  month* 
latafl  pa**  all  prefiring  checkout  tests  with  only  minor  adjustment* 
or  component  replacement*  by  battalion  or  battary  personnel.  Ve 
emphasise  that  these  preflrlng  checks  are  those  performed  in  the  battery 
area  after  the  missile  has  been  carried  or  stored  as  part  of  the 
artillery  unit  basic  load.  They  are  net  the  teats  performed  when  the 
umunltlon  is  issued  at  the  ammunition  supply  point,  nor  tests  performed 
by  the  Ordnance  direct  support  company.  The  adjustments  and  component 
replacements  must  be  done  by  Artillery  soldiers  in  the  unit  without  the 
assistance  of  Ordnance  personnel  and  without  the  use  of  special  equip¬ 
ment  other  than  that  issued  to  the  Artillery  unit.  Also,  the  time 
required  for  the  performance  of  these  functions  must  not  cause  the 
system  to  fall  to  fire  at,  or  reasonably  close,  to,  the  desired  tin* . 

Next,  ve  might  require  -  again  typical  -  that  of  the  missiles 
which  pass  preflrlng  checkout  tests,  not  more  than  5  percent  may  fail 
to  fire  at  the  designated  time  due  to  system  malfunction  throughout 
a  72-hour  period  following  checkout. 

Another  typical  requirement  la  that  those  missile*  which  launch 
must  have  an  ln-fLlght  reliability  **  95  percent.  This  is  the  degree 
of  assurance  that,  after  firing,  the  mlaaile  will  not  abort  due  to 
missile  failure  but  will  deliver  Its  payload  accurately  and  that  the 
warhead  operates  as  intended.  In  battle  ve  would  not  treat  in- flight 
reliability  independently  of  tactical  reliability. 

Our  missile  systems  are  prime  battlefield  targets  and  cannot 
be  left  in  position  for  long  periods  of  tins  -  particularly  nuclear 
dollvory  systems.  Consequently,  we  nay  require  that  only  s  abort  time 
shall  be  required  to  emplace  tbs  system  and  to  fire  the  first  missile. 
This  time  must  Include  the  preflrlng  checkout  of  the  missile  system. 
Though  we  would  like  to  treat  missiles  as  we  do  rounds  of  cannon 
artillery  amnunition,  experience  indicates  that  at  the  present  we 
definitely  need  thle  checkout  just  prior  to  firing. 

let’s  review  cur  required  characteristics.  The  Artillery  expects 
to  receive  an  cpernble  missile  at  the  ammunition  supply  point.  We 
expect  to  be  able  to  store  and  transport  this  ammunition  as  part  of 
our  basic  load.  Using  our  typical  percentages,  90  percent  of  these 
missiles  must  p-ana  preflrlng  checkout  tests  using  standard  army  pro¬ 
cedures  and  average  soldiers  with  no  Ordnance  support  in  the  firing 
area.  95  percent  of  the  checked  out  missiles  must  launch  on  time 
throughout  a  7^-hcur  period  after  checkout.  Then,  95  percent  of  these 
missiles  must  not  abort,  and  must  deliver  their  payloads  on  target 
within  the  nomnl  dlrjvir elon  limits  and  function  properly  upon  arrival. 
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The  reliability  at  checkout,  firing,  and  in-flight  mat  ba  suecasa- 
lrely  realised  in  a  tactical  situation,  i  missile  aystea  which  cannot 
ruooesalvely  pass  each  requirement  is  not  tactically  effective,  ainoe 
it  cannot  complete  lta  fire  mission, 

ihe  user  can  accept  no  relaxation  of  requirements  pertaining  to 
timeliness  of  fire  or  of  infixing  checkouts.  However,  if  such  require¬ 
ments  impose  unacceptable  limitations  ouch  as  excessive  basic  load  of 
apart  components,  or  missiles,  or  requirements  for  exoesslve  personnel 
to  perform  the  prefiring  checkout,  we  net  be  informed  so  that  we  oem 
consider  what  requirements  may  be  relaxed. 

It  would  appear  that  a  great  increase  In  overall  reliability  earn 
be  aecompliabed  in  the  area  of  p  re  firing  checkout.  The  prefixing  check¬ 
out  reliability  may  be  increased  by  the  provision  of  a  level  of  spares, 
based  on  design  and  experlanoa,  ltodlatelv  available  to  the  using  unit* 
i'ila  will  enable  us  to  receive  a  fire  order  a  reasonable  time  before 
firing,  make  the  necessary  component  replacements  without  delaying  for 
Ordnance  support,  and  complete  the  fire  mission.  To  do  this,  the 
missile  system  must  be  designed  eo  that  average  soldlera  with  average 
training  can  determine  malfunctions  and  replace  components  with  little 
difficulty. 

To  summarize, 

1.  Tactical  reliability  can  only  be  defined  after  operational 
requirements  are  known  for  e  particular  system.  Priori  tic  a  of  char¬ 
acteristics  play  a  major  role  in  this  regard. 

2.  The  designer  suet  consider  the  users  "time  to  renal r1  end 
"equipment  required  to  repair"  in  the  developing  system  reliability. 

3.  Averare  soldiers  with  considerable  variety  in  background  and 
training  will  oporn'e  cur  systems. 

Tactical  reliability  must  include  ora  firing  checkout  requirements 
and  maintenance  rcouire,v,i ts  as  well  as  the  usual  "in-flight"  reliability. 
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WHAT  RELIABILITY  MEANS  TO  SAC 
Lt.  Colonel  J.  W.  Andereon,  Jr.  --  Headquarters  SAC 

"'•«  reliability  of  strategic  weapon  systems.  Including  missiles, 
of  the  Strategic  Air  Cotmaand  is  of  great  concern  to  the  Cosnander 
In  Chief  and  planners.  It  is  not  enough  to  just  say  that  it  ia 
desirable  to  have  reliable  equipment.  Neither  is  a  design  goal  of 
high  reliability  Just  an  objective  or  number;  it  must  eventually  be 
demonstrated  and  become  a  valid  planning  factor  upon  which  we  can 
depend.  We  have  these  planning  factors  available  from  the  hundreds 
of  thousands  of  aircraft  sorties,  but  the  missile  systems  pose  an 
entirely  different  proposition.  In  spite  of  the  difficulties  in 
forecasting  missile  capability,  the  numbers,  types,  basing,  and 
the  targeting  of  the  missile  force  is  dependant  upon  such  a  fore* 
cast.  Therefore,  lt  is  not  only  necessary  to  have  a  reliability 
factor,  but  lt  oust  be  sufficiently  demonstrated  to  establish 
proven  confidence  levels. 

Confidence  can  ba  more  vital  than  basic  reliability.  To  over¬ 
simplify  this;  for  targeting  you  would  prefer  a  1001  confidence 
level  in  low  reliability  to  a  lev  confidence  in  100X  reliability. 
Whatever  the  reliability,  you  sust  know  it,  take  lt  into  considera¬ 
tion  in  your  plans,  and  then  set  about  bringing  lt  to  the  level  of 
effectiveness  required.  Improvement  of  reliability  after  a  system 
is  in  the  field  is  usually  a  most  expensive  complicated  program. 

The  ECP  and  mod-program  costa  for  limited  Improvement  to  be  gained 
have  to  be  weighed  against  the  effort  necessary  to  obtain  a  whole 
new  issue.  It  is  usually  sure  practical  to  lnltlatesystem  develop¬ 
ment  with  a  proper  emphasis  on  an  "Optimum  allocation  of  funds  for 
reliability  programs  of  guided  missiles."  That  last  phrase,  by 
the  way,  la  the  title  of  a  paper  by  Dr.  Pleruschka  (with  the  Redstone 
Arsenal)  back  In  1955. 

In  large  ballistic  missiles  you  have  faced  problems  of  precise 
oontrol,  clmln,  ,  metexlng,  and  other  factors  that  lead  to  lntagrated 
system  complexity.  This  was  initially  accompaniad  by  a  general  lack 
of  experience  and  reliability  data.  There  have  been  many  studlas 
and  papers  on  the  subject  of  reliability  programs  necessary  to 
develop  the  weapon  systems,  and  there  will  be  some  fine  presentations 
by  individuals  during  this  symposium  as  well.  But  if  we  are  ever  to 
know  what  our  military  capability  is,  we  must  exercise  the  full 
system  in  the  field  and  eventually  fire  the  missiles.  While  a  launch 
from  an  operational  site  is  desirable,  as  yet  SAC  has  not  had  the 
approval  to  do  this.  At  present  flight  demonstrations  most  be 
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conducted  at  Vandanberg  Air  Force  Base  Into  the  Pacific  Missile 
Range,  or  at  Patrick  Air  Force  Baae  on  the  Atlantic  Hiaaile  Range. 

When  discussing  the  minimum  number  of  each  type  to  be  exercised, 
we  must  consider  atatiatical  validity,  and  of  course,  some  of 
these  oust  be  launched  each  year.  Spending  too  little  for 
^liability  demonstrations  is  a  false  economy;  failure  to  establish 
precisely  correct  confidence  levels  is  militarily  dangerous. 

Now,  if  we  would  dlscuas  specific  factors,  some  definitions  ere 
probably  in  order.  Some  you  will  recognise,  but  where  the  connote* 
tlon  differs,  the  user's  ideas  will  be  presented. 

Overall  Operational  Reliability  of  a  missile  weapon  ayatem  la 
the  probability  that  once  the  execution  order  is  received,  end  the 
commit  sequence  button  pressed,  the  missile  will  be  launched  and 
its  warhead  will  explode  with  design  yield  on  the  specified  target, 
within  the  CEP  definition.  This  may  vary  somewhat  from  R&D 
reliability,  for  performance  within  a  designed  reaction  time  is  also 
measured,  automatic  sequencing  replaces  manual  operation,  and  the 
environment  is  different.  This  probability  la  expressed  by  a 
decimal  number  between  zero  and  one.  Zero  probability  means,  of 
couree,  no  chance  of  the  event  happening;  0.5  probability  means  a 
501  chance  of  the  event  happening;  and  a  probability  of  1.0  mean* 
the  event  la  certain  to  happen. 

Measured  Reliability  is  the  "success  ratio"  obtained  in  a  series 
of  tests;  in  other  words,  the  number  of  successful  tests  divided  by 
the  total  number  of  tests. 

Confidence  Level  is  the  probability  that  the  true  reliability 
lies  between  two  limiting  values,  or  somewhere  above  a  certain 
value.  A  confidence  level  may  be  thought  of- as  a  probability  of 
a  probability. 

Confidence  Limits  are  the  limiting  values  between  which  the  true 
reliability  will  be  for  a  given  confidence  level.  These  limits  depend 
on  the  number  of  measurements,  or  sample  alee. 

The  overall  operational  reliability  factor  used  by  the  Strategic  Air 
Command  for  a  particular  missile  weapon  system  la  not  easily  ascer- 
talned.  It  is  recognized  that  flight  reliability  of  the  missile 
depends  on  reliabilities  of  the  many  individual  sub-ayatema,  auch 
as  propulsion,  flight-control,  guidance  and  re-entry  sub-systems. 
These,  in  turn,  arc  affected  by  the  many  component  reliabilities. 

In  addition,  the  propellant  loading  systems,  erectors,  shelter 
doors,  operating  consoles,  cabling,  etc.,  are  all  Involved.  Over¬ 
all  reliability  then,  is  Influenced  by  many  individual  probabilities. 


including  those  subject  to  personnel  fee  tors,  ground  support  equip* 

Bent  (CSC)  and  the  aiseile  Itself. 

7b*  complexity  of  the  problea  to  determine  the  reliability  of  a 
a-iiiiic  weapon  system  is  further  complicated  by  the  difficulty  of 
making  aaasureaeuts  under  actual  operating  conditions.  For  instance, 
a  system  davelopad  in  Florida  and  demonstrated  in  California,  does 
not  necessarily  give  SAC  a  true  measure  for  e  Wyoming  baaed  alaelle. 
in  some  Instances,  there  la  e  tendency  to  make  gross  extreoolatlons 
from  reliability  factors  determined  under  research  and  development 
conditions  to  use  as  operational  planning  factors.  It  should  be 
realised  that  we  moat  endeavor  to  get  test  results  under  conditions 
as  near  to  actual  operating  conditions  as  possible,  not  only 
geographic,  but  climatic  as  well.  By  conducting  some  teats  on  a 
"Ho  notice"  basis  the  pressures  on  personnel  and  equipment  that  are 
likely  to  occur  in  time  of  conflict,  may  be  duplicated  in  o.*der  to 
obtain  useful  and  valid  data. 

The  number  used  for  overall  operational  reliability  of  a  mlsalle 
weapon  system  la  not  simply  the  product  of  all  applicable  measured 
success  ratios.  Nor  can  a  theoretical  extrapolation  of  the  product 
of  these  smasurea  values  be  used  to  obtain  a  realistic  success  ratio 
to  be  expected  under  anticipated  operating  conditions.  In  arriving 
at  the  value  for  the  overall  operational  reliability  of  e  weapon 
system,  confidence  limits  associated  with  measured  succeea  ratios 
must  be  taken  into  consideration.  These  confidence  limits  are  based 
on  an  assumed  binominal  distribution  for  test  results.  To  illustrate 
confidence  limits,  consider  the  following  exaople:  Suppose  a  measured 
reliability,  or  success  ratio,  of  O.SO  was  obtained  in  testing  10 
missiles  fired  under  as  realistic  operating  conditions  as  possible. 

The  true  reliability  of  similar  missiles  fired  under  the  same  con* 
dltlons  would  be  somewhere  betveen  27X  and  73X,  at  the  SOT  confidence 
level.  The  true  reliability  would  lie  between  19X  and  411,  at  the 
95X  confidence  level.  If  the  0.50  measured  success  ratio  had  been 
obtained  in  testing  20  missiles  Instead  of  10,  fJ:he  true  reliability 
would  lie  between  0.34  and  0.64  at  the  80X  confidence  level,  and 
between  0.27  sod  0.73  at  the  957,  confidence  level.  The  inter¬ 
relation  of  demonstrated  reliability,  confidence  factors  and  sample 
size  is  familiar  to  all  of  you. 

A  realization  of  what  effect  reliability  has  on  force  size  may  be 
realized  by  noting  chat  we  will  not  receive  authority  to  expend  our 
missile  force  untfl  alter  an  enemy  strike  has  been  initiated.  It 
would  seem  reasonable  that  this  strike  would  have  been  planned  for 
90  or  957.  probability  of  success.  This  leaves  ua  in  the  position  of 
attempting  deterrence  with  those  surviving,  maybe  5  or  10X  of  the 
total  force.  This  tcroinlng  force  is  further  degraded  by  ln-commlaslon 
rate*  reliabll tty . 
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As  an  example  of  wha  t  reliability  Chen  means  In  terms  of  surviving 
missiles  after  an  enemy  attack,  let  us  assume  that  we  require  951 
probability  of  destruction  of  200  separate  enemy  targets.  Let  us 
further  assume  for  simplicity's  sake,  that  we  have  mlaalles  of  a 
Cyp-  that  one  delivered  warhead  would  destroy  a  target.  Then,  400 
surviving  missiles  with  80%  overall  operational  reliability  would 
do  the  Jot.  But  If  the  reliability  was  only  40%,  1200  surviving 
missiles  would  be  required  to  destroy  the  200  targets.  This  is  a 
difference  of  800  surviving  missiles,  or  several  thousand  additional 
missiles  of  the  less  reliable  type  required  for  original  Inventory 
that  must  be  manned  and  maintained. 

The  importance  of  reliability  of  missile  weapon  systems  Is  perhaps 
most  striking  In  a  situation  where  it  Is  of  utmost  Importance  that 
particular  aiming  points  be  destroyed,  and  the  measured  reliability 
Is  based  on  a  small  number  of  firings.  Assume  our  missiles  to  have 
5-KT  yield,  and  1-NM  CEP.  (CEP  Is  defined  as  the  radius  of  a  circle 
about  the  aiming  point  in  which  half  of  the  missiles  are  expected  to 
fall.)  Now,  consider  a  situation  where  our  missiles  are  targeted 
against  weapon  systems  so  protected  that  10  psl  overpressure  would  be 
required  to  destroy  them.  Let  us  assume  again  that  a  951  probability 
of  destruction  is  desired.  A  delivered  warhead  will  destroy  its 
target  In  this  example,  but  the  lack  of  confidence  In  measured 
reliability  of  our  missiles  requires  more  than  one  missile  per 
target.  Consider  two  cases:  the  measured  reliability  is  0.40  In 
one  case,  and  0.80  In  the  other,  each  determination  being  based  on 
20  firings  of  missiles  in  ss  nearly  an  operational  environment  aa 
possible.  With  901  confidence,  we  know  that  the  true  reliability 
is  equal  to  or  greater  chan  some  value  lower  than  the  measured  one. 

In  the  first  case  this  lower  limit  Is  0.25,  and  in  the  second  case, 
0.65.  Using  these  lower  limits  of  rellabl  lty  as  planning  factors, 

10  of  the  less  reliable  missiles  per  aiming  point  are  required; 
while  only  3  per  aiming  point  are  required  of-the  more  reliable 
missiles. 

The  effects  of  missile  reliability  and  the  confidence  with  which 
It  is  known  are  even  more  pronounced  for  hardened  targets.  In  the 
preceding  example,  had  Che  target  been  able  to  withstand  over¬ 
pressures  up  to  100  psi,  one  could  not  assume  that  one  of  the  same 
delivered  weapoi s  would  destroy  that  target.  Now  it  would  require 
25  of  the  missl.es  of  0.40  reliability  to  have  a  951  probability  of 
destruction,  while.  9  of  the  missiles  of  0.80  reliability  would 
suffice  for  the  specified  907.  confidence  in  both  cases.  This  Is  a 
difference  of  lb  missiles  per  aiming  point  scheduled  in  the  targeting 
plan.  Please  realize  that  the  examples  I  have  used  are  not  indicative 
of  any  true  capability  or  plan;  they  are  for  illustration  only. 
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Having  posed  the  user's  problem  of  determining  reliability  and 
confidence,  and  the  importance  of  this  determination  in  his  planning 
factors,  we  should  look  at  the  solution  to  the  problem.  Air  Force 
Regulation  80*14  defines  three' categories  of  weapon  system  testing: 

Category  I  -  Subsystem  Test.  This  is  the  shake  -  rattle  -  roll, 
hot*blowing  fungus,  etc.,  with  which  the  manufacturers  are  all 
familiar. 

Category  II  -  Integrated  System  Test.  In  missile  weapon  systems 
this  should  include  the  CSE  and  facility  as  well.  A  military  caps* 
blllty  la  also  implicit  in  this  phase.  This  is  usually  met  by 
integrating  the  using  conmand  unit  personnel  into  the  test  program 
in  ever  increasing  numbers  and  with  increasingly  greater  responsibilities. 

Category  III  •  Operational  System  Test.  This  is  the  user's  responsi¬ 
bility,  though  hewill  bn  joined  by  the  conaands  responsible  for  the 
various  inputs  to  the  whole  system.  ARDC  As  the  developer,  AMC  as 
logistic  support  manager,  and  ATC  as  the  personnel  pipeline  manager. 

This  may  be  likened  to  an  Inspector  General  giving  the  new  commander 
a  report  on  the  unit  he  has  just  Inherited. 

In  addition  to  participation  in  test  and  evaluation  of  the  system  as  it 
has  been  Issued,  the  user  has  another  fine  aource  of  data  to  help  him 
"figure"  his  capability.  This  sor  :t  is  his  exercise  of  the  system 
during  unit  training  (we  call  this  integrated  weapon  system  training, 
IUST),  during  the  ORI  (operational  readiness  inspection),  normal  SAC 
"No-notice"  alerts,  and  the  recurring  combat  training  launches  each 
year.  Youwill  remember  that  we  do  not  now  have  permission  to  launch 
from  all  operational  sites,  and  therefore  the  validity  of  the  data 
must  be  adjusted  somewhat. 

Gentlemen,  we  have  looked  at  the  user's  requirement  for  high  confidence 
in  a  known  reliability  and  the  effect  on  his  force  size,  desires  for 
nod-programs,  basing,  manning  and  operational  planning.  The  Air  Force 
has  a  user  reliability  program  in  effect,  so  thst  he  may  have  con¬ 
fidence  in  a  demonstrated  reliability.  I  am  sure  tha  manufacturers  and 
developers  will  have  soioe  Interesting  things  to  tall  ua  In  the  next 
two  and  a  half  days  about  how  to  meet  those  requirements  in  the  most 
efficient  way.  This  symposium  has  promise  of  being  a  very  Interesting 
experience . 


INTERSERVICE  DATA  EXCHANGE  PROGRAM 
(1DEP) 


J.  H.  Draughon,  ABMA 


In  this  age  of  budget-buster  missile  and  space  programs,  the 
majority  of  which  are  proceeding  on  a  crash  basis  and  all  of  which 
are  pushing  the  state  of  the  art,  it  was  inevitable  that  the  constant 
search  for  components  of  higher  quality  and  greater  reliability  would 
lead  to  much  duplication  in  test  programs  throughout  the  Armed  Forces 
and  industry.  -The  program  I  shall  outline  to  you  presently  has  as  its 
primary  objective  the  reduction  of  that  duplication  of  effort  in  the 
ballistic  missile  and  space  programs,  so  that  those  dollars  and  those 
scientific  and  technical  manhours  saved  stay  be  more  effectively  applied. 
The  program  is  known  as  "IDEP"  -  Interservice  Data  Exchange  Program. 


1  do  not  think  IDEP  will  necessarily  reduce  the  coat  of  testing. 

I  do  believe  it  will  result  in  broader  end  better  test  programs  for 
the  same  dollars  and  In  an  enhancement  of  ballistic  missile  and  apace 
vehicle  reliability.  If  the  exchange  of  these  data  results  in  the 
detection  and  replacement  of  a  single  unreliable  or  marginal  component, 
and  Just  one  large  ballistic  missile  or  space  vehicle  la  saved,  the 
program  will  not  have  been  in  vain. 


IDEP  had  Its  Inception  as  a  result  of  the  airing  of  mutual  problems 
in  quality  control  and  reliability  in  a  series  of  meetings  among 
representatives  of  the  three  Armed  Services  --  Army,  Navy  and  Air  Force, 
The  first  of  these  meetings,  in  early  1958,  was  sponsored  by  the  Air 
Force  Ballistic  Missile  Center  at  Inglewood,  California.  A  second 
meeting  in  December  1958,  sponsored  by  what  was  then  the  Navy  Bureau 
of  Ordnance  Special  Projects  Office,  was  held  at  Lockheed's  Sunnyvale, 
Calif.,  plant.  In  July  1959,  Chrysler  Corp.  Missile  Division  in 
Detroit  was  host  to  a  meeting  sponsored  by  the  Army  Ballistic  Missile 
Agency. 


Prom  these  and  other  interim  meetings,  a  Coordination  Croup  was 
formed,  composed  of  one  member  each  from  the  Air  Force  Ballistic  Missile 
Division,  the  Air  Force  Ballistic  Missiles  Center,  the  Navy  Bureau  of 
Ordnance  Special  Projects  Office,  and  the  Army  Ballistic  Missile  Agency. 
This  group,  of  which  ]  am  presently  the  Chairman,  generated  a  proposal 
for  the  exchange  of  user  component  test  data  among  ballistic  missile 
contractors  of  the  AFBMD ,  AFRMC,  Navy  Special  Projects  Office,  and 
ABMA.  (See  Figure  1  -  Flow  ol  Information  to  Data  Centers  and  out  to 
Contractors,  otr.) 
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Mow,  the  exchange  of  date  among  the  Services  is  not  a  new  Idea. 

In  fact,  there  has  been  a  constant  exchange  of  test  data  among  the  Air 
Force,  Army  and  Navy  on  an  “as  requested"  basis.  This  proposal  is 
unique  in  that  it  goes  several  steps  further  and  provides  for  an 
auh—itic-  exchange  of  user  component  test  data  among  selected  contractors 
without  the  necessity  to  publish  Hats,  make  official  formal  requests, 
staff  approval  of  such  requests,  and  finally  mall  the  requested  data. 

Before  we  go  any  further,  let  me  define  this  word  "component". 

As  I  will  use  it,  a  component  is  defined  as  a  part  or  place  not  normally 
subjected  to  further  disassembly,  e.g.,  capacitors,  resistors,  relays, 
etc. 

The  importance  ot  and  the  urgent  need  for  a  better  means  of 
exchanging  information  and  data  in  the  missile  and  space  business  has 
been  recognized  by  various  public  and  private  management  officials  and 
by  Congressional  committees,  not  the  least  of  which  is  the  House  Appro¬ 
priations  Committee. 

Since  there  can  be  no  question  of  the  desirability  of  such  a 
program,  let  us  examine  the  problem.  Among  other  things,  investigation 
has  shown  that  in  one  specific  instance  eight  different  organizations 
had  performed  certain  controlled  laboratory  tests  on  components  of  the 
same  manufacture  and  part  number.  As  it  turned  out,  the  results  from 
any  one  of  the  teats  would  probably  have  satisfied  the  requirements  of 
the  other  seven  if  they  had  been  aware  of  the  test  results.  If  such 
duplication  of  effort  can  be  stopped,  we  will  save  not  only  the  teat 
setup  costs  but  also  the  time  of  the  test  and  design  engineers  and 
technicians.  With  the  application  of  the  saved  effort  to  other  urgent 
problems,  an  acceleration  of  development  programs  should  ba  realized. 

Further,  broader  and  more  prompt  access  to  controlled  laboratory 
test  data  will  undoubtedly  point  the  way  to  better  manufacturing  and 
quality  control  procedures  which,  in  turn,  should  result  in  a  higher 
level  of  quality  being  maintained  within  the  design  specification- 

Now,  there  may  be  those  who  feel  that  when  component  quality  Is 
improved,  system  reliability  is  slso  improved.  This  is  not  necessarily 
true.  If  a  uroig  pirt  is  specified  by  the  system  designer,  no  amount 
of  diligence  by  the  manuincturer  nor  by  his  quality  control  organization 
will  make  the  missile  system  any  more  reliable  than  that  part  is 
reliable  in  its  poitlcolm  environment.  However,  if  we  can  make  avail¬ 
able  to  the  nytitem  i.'eaigimr  component  performance  or  failure  data  which 
points  up  the  in  id  Un  a  change  to  a  component  of  higher  reliability, 
enhanced  system  t e 1  in h L l i ty  should  result.  The  need  for  higher  quality 
and  more  reliable  <  .  ni  one:-*. 9  cannot  be  overemphasized. 

If  a  trnnsintm  falls  in  an  automobile  radio,  at  most  the  owner 
is  irked  hiuJ  p>«  ■’  e  •>  llt.tii  Inconvenienced  while  it  is  being  repaired. 


But  if  a  translator  fails  in  a  missile  or  space  vehicle,  a  multimillion 
dollar  loss  may  be  incurred.  We  could  loss  the  Whole  vehicle,  or  we 
could  lose  a  part  or  all  of  the  information  the  vehicle  was  launched 
to  secure.  In  any  event,  the  loss  is  considerable. 

Now,  let's  take  a  look  at  the  different  types  of  information  which 
might  be  exchanged.  Some  of  these  are; 

1.  User  component  qualification  test  data,  l.a.,  laboratory- 
controlled  test  results; 

2.  Quality  control  test  results; 

3.  Reliability  test  results; 

4.  Exaggerated  stress  test  results;  and 

5.  Flight  test  results,  - 

Tou  will  recall  that  I  stated  earlier  that  a  proposal  had  been 
generated  for  the  exchange  of  user  component  test  data.  This  element 
of  the  various  existing  types  of  data  was  selected  because  It  appeared 
that  this  type  of  data  would  lend  Itself  more  readily  to  Immediate 
exchange  and  the  benefits  would  be  immediate  and  extensive.  Such  test 
reports  will  specify  that  a  given  component  has  been  tested  under 
controlled  environmental  conditions,  and  that  the  results  of  such  tests 
are  as  stated  in  the  report.  If  a  designer  has  need  for  such  a 
component  for  a  given  environment  and  he  has  available  to  him  Che  results 
of  user  tests  run  under  the  same  or  similar  environment,  he  should  be 
able  to  make  a  selection  with  reasonable  assurance  and  without  further 
testing,  or  at  least  with  a  minimum  of  additional  testing.  If  the  tests 
to  which  he  has  access  supply  only  a  part  of  the  information  which  he 
requires,  we  are  atiLl  ahead  of  the  game  inasmuch  as  only  partial  tests 
need  be  run  to  prove  the  component  for  his  particular  application. 

In  the  case  of  the  exchange  of  quality  control  test  data,  since 
the  volume  of  such  information  Is  extremely  large,  Individual  test 
results  would  not  be  exchanged.  We  contemplate  that  periodically  an 
analysis  of  quality  control  tests  would  be  made  and  the  results  of  the 
analysis  would  be  Incorporated  in  the  data  exchange  system. 

Now,  flight  tost  results  pose  quite  a  different  problem.  Thera 
is  such  a  wide  difference  In  the  environment  encountered  in  the  flight 
of  the  8ame  coirij^iunt  in  different  missile  systems  (in  many  instances 
the  exact  environment  ir<  not  known  or  may  vary  from  flight  teat  Co 
flight  test),  that  It  becomes  almost  impossible  to  make  any  analysis 
of  the  given  component  which  wo  jld  be  meaningful  to  other  missile 
systems.  For  e*.r  :  le,  a  given  relay  might  encounter  a  vibration 
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characteristic  in  Missile  A  of  350  cycles  per  second.  Ths  same 
component  if  used  in  Missile  B  might  encounter  a  vibration  characteristic 
of  half  this  value.  While  in  still  another  missile  it  might  encounter 
a  characteristic  of  twice  that  factor.  Even  at  different  locations 
within  the  same  missile,  the  vibration  characteristic  is  apt  to  be 
different.  So,  to  earmark  a  given  component  as  unfit  for  missile  use 
because  of  a  failure  in  one  missile,  is  completely  unrealistic. 

However,  if  failures  are  such  that  a  significant  pattern  is  formed, 
t't'ese  data  will  be  exchanged. 

Experience  has  shown  that  we  can  say  that  a  given  component  la 
satisfactory  or  unsatisfactory  for  use  in  a  given  missile  only  when 
the  environment  in  which  the  component  will  function  la  known.  It, 
therefore,  follows  that  the  only  meaningful  measure  of  e  component  ie 
its  ability  to  perform  within  lta  design  characteristics  under  specific 
environmental  conditions.  We  will  attempt  to  see  that  the  date  exchanged 
under  this  program  meet  this  requirement. 

Needless  to  say,  one  of  the  most  difficult  hurdlea  to  a  meaningful 
exchange  of  technical  test  data  is  s  common  language  which  la  under¬ 
standable  to  all.  By  this,  1  mean  some  sort  of  coding  ao  that  a  given 
environment  ie  always  expressed  the  same,  or  that  a  given  type  of 
failure  is  always  expressed  in  the  same  terms.  Much  work  haa  been  done 
by  the  group  working  on  this  proposal  for  the  exchange  of  data  on  such 
a  language.  As  the  takeoff  point,  tha  coding  which  had  bean  worked  out 
for  the  TITAN  missile  system  was  used.  This  has  been  supplemented,  and 
we  feel  thet  we  have  at  least  the  beginnings  of  a  language  which  could 
be  used  and  would  be  understandable  to  all.  (Sea  Pigura  2  -  Sample  of 
Coding) 

Probably  the  greatest  single  problem  in  this  exchange  program  la 
Che  method  of  transmitting  the  dats  from  one  point  to  another  in  such 
a  manner  that  it  is  readily  retrievable.  As  you  all  know,  there  are 
masses  and  masses  of  data  which  heVe  been  generated.  I  would  not  even 
hazard  e  guess  as  to  the  tons  of  recorded  Information  which  haa  been 
generated,  in  fact,  the  major  question  becomes  on#  of  how  can  wa  take 
a  mass  of  Information  and  reduce  it  to  proportions  which  can  ba  handled, 
and  at  the  same  time  make  that  information  easily  available  or  retriev¬ 
able  to  the  engineer  who  needs  it?  This  problem  has  been  the  basis 
of  many  CAM  and  computer  program  studies. 

For  IDEP  application,  we  beLieve  that  the  answer  la  contained  in 
the  Military  "D"  double-aperture,  microfilm  card.  (Sea  Pigura  3  - 
EAM  Card)  At  a  reduction  ratio  of  22-to-l,  two  pieces  of  35  n*n  micro¬ 
film  affixed  to  the  two  apertures  can  record  eighteen  (18)  pages  of 
written  text.  By  making  a  survey  of  a  substantial  manber  of  test 
reports,  we  have  found  that  L8  pages  would  cover  about  90X  of  all  the 
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u4«r  component  casta  conducted  under  controlled  laboratory  condition*. 
The  remaining  101  can  be  cover ad  by  a  second  or  third  follower  card. 
Teats  using  Military  "D"  double-aperture  card*  have,  shown  that  no 
■edification  to  standard  Machinery  la  required  for  aatlsfactory  Machine 
process log. 

Now.  with  the  means  of  raduclng  the  large  mats  of  data  to  propor¬ 
tions  which  can  ba  handled,  the  next  question  becomes  one  of  who  does 
the  handling?  It  has  been  decided  that  this  could  best  be  done  by 
establishing  what  wa  call  Control  Canters,  or  Data  Distribution  Centers, 
at  three  different  points.  One  will  ba  at  AXMA  in  Buntevllle.  another 
will  be  at  the  AFBMD  complex  in  the  Los  Angelas  area,  and  the  other 
will  be  at  tha  Naval  Ordnance  Laboratories  at  Corona,  California.  All 
AJhA  ballistic  missile  contractors  will  faad  laboratory  tast  reports, 
ccdo>i  to  tha  conaon  language,  to  tha  AJKA  Data  Distribution  Center, 
where  they  will  he  microfilmed  and  the  film  placed  in  the  aperture  cards 
wh"ch  have  been  keypunched  to  permit  machine  sorting.  The  required 
nuri>er  of  copies  will  ba  made  from  the  master  card  by  a  eard-to-card 
duplicating  process.  Tha  aperture  cards  will  then  be  distributed 
directly  to  the  Air  force  and  Navy  Data  Distribution  Centers,  their 
designated  contractor*,  and  other  ABMA  contractors.  The  Air  Pores  and 
Navy  Cantare  will  operate  on  tha  seme  basis.  All  Information  will  not 
flow  to  all  contractors;  in  other  words,  the  designation  of  contractors 
will  be  on  a  specified  area-of-lnteraat  basis  by  the  Services  concerned. 
Information  in  a  given  contractor's  particular  area  of  Interest  will 
flow  to  that  contractor,  but  ha  will  not  receive  other  information 
which  is  not  pertinent  to  hla  particular  area  of  Interest.  Copies  will 
also  go  to  the  Armed  Services  Technical  Information  Agency.  Qualified 
contractors  and  other  agendas  not  a  part  of  IDEP  can  thus  secure  tha 
same  information  by  proper  request  to  AST LA.  At  the  same  time  tha 
aperture  card  is  prepared,  a  5"x8"  printed  summary  card  will.be  prepared 
from  a  sumeary  page  contained  in  each  teat  report.  (See  Figure  3  - 
Report  S ternary  Card)  This  sunenary  card  will  glva  pertinent  summarised 
information  on  the  test  so  that  a  designer  can  determine  applicability 
of  the  teat  without  having  to  etudy  the  entire  report.  Thaaa  summary 
cards  will  be  distributed  along  with  the  aperture  cards.  An  index  of 
all  reports  distributed  will  be  prepared  and  distributed  on  a  monthly 
basis. 

Of  course,  this  1  all  very  nice  to  have  a  lot  of  cards  in  a  Data 
Distribution  Center  and  other  cards  sent  out  to  various  contractors, 
but  tht  big  question  is,  what  happens  to  them  after  they  gat  thara 7 

The  objective  is  to  put  these  carde  immediately  before  tha 
engineers  needing  the  infonration,  Obviously,  v  can't  put  all  carde 
before  all  engineers,  but  with  a  little  Judicious  rganlsatlon  wa  can 
establish,  at- specific  strategic  points  within  eacu  contractor's  plant 
and  within  our  own  service  laboratories,  decks  of  tbaet  cards  which 
carry  test  results  pertaining  to  the  specific  area  of  interest  of  the 
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group.  AC  these  strategic  locations  will  ba  locatad  vlewera.  After 
a  aelaction  is  made  from  the  5x8  aunmary  cards ,  the  aperture  card  la 
placed  In  the  viewer,  and  one  can  read  the  text  of  the  report  blown 
hack  to  tha  original  Latter  aiaa.  This  givaa  the  engineer  the 
capability  of  a  quick  look  at  tha  report.  If  he  neade  to  make  a  detailed 
study  of  tha  elements  of  tha  report,  ha  may  then  call  for  a  hard  copy. 
This  can  be  accomplished  by  one  of  tha  high-speed  duplicating  processes 
njw  on  the  market.  These  are  capable  of  reproducing  in  let ter- else 
six  cards  per  minute  per  aperture  at  a  coat  of  approximately  15c  per 
aperture,  including  labor  to  cut  reports,  etc.  I  have  a  hard  copy  made 
by  one  of  tha  processes  after  a  card  had  undergone  1000  peases  through 
mcchine  sorters,  collators,  etc.  The  text  is  perfectly  legible.  If 
the  contractor  doea  not  have  high-speed  duplicating  equipment,  any 
photographic  shop  can  blow  the  microfilm  back  to  legible  alts. 
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The  scope  of  this  program  has  been  held  to  ballistic  miaslle 
contractors  of  the  Army,  Air  Force,  and  Navy,  in  order  to  work  out  the 
bugs  in  what  we  Relieve  to  ba  a  good  systaa.  If  .the  trial  run  is  as 
successful  as  we  have  reason  to  believe  it  will  be,  we  will  propose 
that  IDEP  be  expanded  to  embrace  other  areas. 

Now,  what  about  costa?  I  won't  say,  "Nothing,"  because  1  don't 
know  of  anything  that  la  free  in  this  day  and  age.  I  will,  however, 
point  out  certain  pertinent  facts: 

a.  The  equipment  for  doing  this  Job  is  tha  same  as  that  which 
will  be  required  in  the  Department  of  Defense  drawing  microfilming 
program  which  applies  to  all  Armed  Services.  It  will  be  available  in 
most  large  contractors  plants. 

b.  The  missile  business  is  a  multi-billion  dollar  business, 
of  which  teatlng  to  determine  suitable  components  for  a  given  environ¬ 
ment  is  no  small  part.  If  we  can  reduce  the  multiple  tasting  of  the 
same  components,  we  will  gain  many  engineering  manhours  which  can  be 
applied  to  advancement  of  the  state  of  the  art.  In  view  of  the  recent 
Russian  successes ,  I  think  there  can  be  little  doubt  of  tha  necessity 
of  using  every  asset  at  our  disposal  to  expedite  our  own  missile  and 
space  efforts. 

c.  If  this  data  exchange  program  results  in  the  change  of  a 

*  component  which  saves  even  one  TITAN,  ATLAS,  THOR,  POLARIS  or  JUPITER 

missile,  to  say  nothing  of  a  SATURN  Space  Vehicle,  the  cost  of  the 
program  will  have  been  paid  for  years  to  come. 

Now,  let  me  leave  you  with  one  parting  thought.  An  ultimate 
objective  of  this  program  is  to  work  out  a  system  of  environmental 
coding  and  a  computer  program  so  that  a  designer  can  query  the  computer 
and  receive  from  the  computer  the  top  components  which  have  been  tested 
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to  the  apeclflc  environmental  Units  required.  When  this  can  be  dona, 
even  store  time  and,  consequently,  manhours  of  effort  will  be  saved.  It 
la  no  sotall  undertaking,  but  wa  have  every  reason  to  believe  it  can  be 
done. 


The  status  of  1DEP  is  as  follows: 

On  September  11,  19 59,  Brigadier  General  Barclay,  Commander 
of  ASHA,  approved  and  forwarded  to  Rear  Admiral  Raborn,  Chief  of  the 
Special  Projects  Office,  Navy  Bureau  of  Ordnance. 

On  September  25th,  Admiral  Raborn  approved  and  forwarded  to 
Major  Ceneral  Funk,  Commander  of  Air  Force  Ballistic  Missile  Center. 

On  October  2nd,  General  Funk  approved  and  forwarded  to  Major 
General  Ritland,  Conmander  of  the  Air  Force  Ballistic  Missile  Division. 

On  October  lAth,  Major  General  Ritland  approved.  (See  Figure  A) 

Since  that  date,  draft  manuals  of  instruction  and  of  the  Generic  Code 
have  been  prepared  for  the  operation  of  the  Data  Distribution  Centers 
and  for  the  guidance  of  participating  contractors.  Meetings  have  been 
held  to  go  over  the  drafts,  and  publication  of  the  manuals  la  lmninent. 
An  approach  to  orient  the  selected  contractors  and  distribution  of 
manuals  to  those  contractors  ia  in  progress  and  will  be  considerably 
expedited  in  the  lnraediate  future.  Many  of  you  who  are  represented 
here  today  are  among  tho&“  who  either  have  been  or  will  be  called  upon 
in  the  near  future.  By  the  next  meeting  of  this  Symposium,  a  definite 
evaluation  of  1DEP  will  be  available. 


Members  of  the  IDEP  Coordination  Group  are: 


Mr,  J.  H.  Draughon,  ABMA 
Mr.  E.  J.  Lancaster,  AFBMC 
LCDR  R.  Smiley,  INSORD 
Major  V.  Brucha,  AFBMD 


Chairman 

Secretary 

Member 

Member, 


Members  of  the  Sub-Group  on  Mechanics  of  Interchange  are: 


Mr,  S.  Follock,  NPL-Corona 
Mr.  C.  A.  Cuthrie,  Aim 
Mr.  M.  Birin- ,  STL,  for  BMD 
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INTERSERVICE  DATA  EXCHANGE  PROGRAM 


REPORTING  ACTIVITY  CODES 
(SAMPLE) 


Ai  General  Electric,  Syracuse,  N.Y. 

A2  General  Electric,  Philadelphia 

A3  Bell  Telephone  Labs,  Whlppany,  N.J. 

A4  A.C.  Spark  Plug,  Milwaukee,  Wise. 

A5  Avco,  Lawrence,  Maaa. 

A6  Martin,  Denver,  Colo. 

A7  AMF,  Pacoima,  Calif. 

A8  Rocketdyne  (North-Amerlcan) ,  Canoga  Park,  Calif. 
A9  Convair,  San  Diego,  Calif. 

B1  Douglas,  Santa  Monica,  Calif. 

B2  Aerojet,  Sacramento,  Calif. 

B3  Arms,  Garden  City,  N.Y. 

B4  Burroughs,  Paoli,  Pa. 

B3  Remington-Rand  Unlvac,  St.  Paul,  Minn. 

B6  Aerojet,  Azusa,  Calif. 

B7  Western  Electric,  Whlppany,  N.J. 

B8  Sycamore  Test  Site 

B9  Santa  Susans  Test  Site 

Cl  Martin  Denver  Test  Site 

C7  AMF,  Greenwich,  Conn. 

7A  Edwards  Air  Force  Base 

7b  Patrick  Air  Force  Base 

7C  Holloman  Air  Force  Base 

7D  Cooke  Air  Force  Base 

8N  NOL,  Corona 

9R  AUHA  (Redstone  Arsenal) 

9S  Martin,  Orlando,  Fla. 

9T  ClirvMier ,  Detroit,  Mich. 

9U  For'!  Inst.  Co.,  New  York  City,  N.Y. 

9V  L*  s  Angeles,  Calif. 
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Figure  2  (Part  1  of  III) 
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INTKRSERVICE  DATA  EXCHANGE  PROGRAM 

COMPONENT  AND  PART  CLASSIFICATION  CODE  INDEX 

(SAMPLE) 


025  Accumulators 

038  Aircraft  Electronic  control  Equip. 
051  Amplifier* 

041  Antennae 

091  Audio  Accessor las 

101  Batteries 

115  Bellows 

121  Blowers  and  Fans 

1M  Boards,  Printed  Circuit 

151  Capacitors,- Fixed 

lol  Capacitors,  Variable 

181  Colls,  Inductance 

191  Computer  Elements 

201  Connectors,  Electrical,  AF  & 

Power 

211  Connectors,  Electrical,  RF  &  Coax. 

241  Crystals,  Frequency  Determining 

301  Electron  Tubes 

305  Electronic  Equipment 

318  Environmental  Simulation  Equip. 

321  Filters,  Electrical 
325  Filters,  Non-Electrical 
335  Fittings,  Pipe 
341  Fuses,  Circuit  Protection, 
Non-Reset. 

345  Gaskets  and  Packings 
358  Cyroscopes 

361  Hardware,  Electro-Mechanical 
(Current  Carrying) 

371  Hardware,  Electro-Mechanical 
(Non-Current  Carrying) 

381  Hardware,  Mechanical 

391  Heatera  and  Coolers,  Electrical 

405  Hydraulic  4  Pneumatic  Parti  &  Comp 

411  Idsntif icstion  Do  ices 

415  Ignition  Parts  and  Pyrotechnics 


428  Instruments,  Gen. .Lab. , 

Controlling 

438  Instruments,  Gen.  Lab.,  Teat 
448  Instruments,  Gsn.  Lab., 

Recording 

451  Inatr  wants,  Indicating 
(Comp.  Pts.  only) 

461  Light  Sources,  Electrical 
491  Magnets,  Permanent 
501  Materials 

531  Motors,  Control  &  Generators, 

Precision 

541  Mounts,  Shock  &  Vibration 
Isolation 

544  Oscillators,  Electrical 
551  Power  Supplies 
565  Propulsion  Components  6  Parts 
575  Pumps  and  Turbines 
588  Receivers ,  RF 
601  Relays 
651  Resistors,  Fixed 
661  Resistors,  Variable 
701  Rotating  Machinery  (Power  Output) 

751  Semiconductor a,  Diodes  and 
Transistors 

771  Solenoids  (Mechanical  Output) 

791  Switches,  Electric,  Automatic 
801  Switches,  Electric,  Manual 
805  Tanks  and  Tank  parts 
808  Telemetry  Components  6  Systems 
811  Timers  and  Counters 
851  Transducers 
901  Transformers 
925  Valves  and  Valve  Parts 
941  Wavegulded  and  Microwave  Plumbing 
951  Wire,  Cable,  and  Harnesses  ; 
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INTERSERVICE  DATA  EXCHANGE  PROGRAM 


COMPONENT  AMD  PART  CLASSIFICATION  CODE 
(SAMPLE) 


141 


151 


Boarda.  Printed  Circuit 


141.00 

ConatructlOQ 

.10 

Double  Sided  (Plain) 

.15 

"  "  (Printed  Component  Parte) 

.25 

Eylatad  Holed 

.65 

Plated  Ho lee 

.75 

Single  Sided  (Plain) 

.80 

"  *'  (Printed  Component  Parte) 

Capacitors.  Fixed 
151.00  Hitt rial 

.05  Ceramic,  General  Purpose 

.07  "  ,  Temperature  Compensating 

.10  Electrolytic,  Aluminum 

.15  ”  ,  Tantalum,  Poll,  Etched,  Non- Polar lead 


.16 

II 

It 

"  "  ,  Polarlaed 

.23 

II 

II 

,  General 

.25  - 

•  1 

«« 

,  Sintered  81ug,  Nun-Polarised 

.26 

•1 

It 

"  "  ,  Polarlaed 

.40 

Glasa 

.45 

Mica,  Poll 

.47 

"  ,  Silvared 

.65 

Paper,  Poll 

.70 

"  ,  Me tallleed 

.75  Paper  -  Plaatlc,  Poll 

.75  "  "  ,  Matalllaed 

.80  Plaatlc,  Poll 

.81  "  ,  Matalllaed 

.95  Vitreous  Enamel 


151.00.00 

.10 

.30 

.35 

.40 

.50 

.70 

.90 


Construction 
Encapeulatad  - 
Hermetically  Sealed,  Can 

"  ••  ,  Rectangular 

"  "  ,  Tubular 

Molded 


Rectangular 

Tubular 
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FOR  BALLISTIC  MISSILES  AND  SPACE  SYSTEMS 


The  recommendation*  made  by  the  Coordination  Group  for  the 
1  nler- Service  Exchange  of  Ballistic  Miaaile  Component  Teat  Data 
have  been  examined  ini  appear  to  be  reaaonable,  deiirxble,  and 
to  the  beat  intereata  of  the  Government. 

Approval  of  the  recommendation*  and  authority  to  implement  in 
my  Command  is  evidenced  by  signature  affixed  hereto. 


Eland 

teraL,  USAF 
ter,  AFBMD 


BEN  I7FXJNK  ' 
Major  General.  USAF 
Commander,  BMC 


W.  F.  RABORN,  JR. 

RADM,  USN 

Director,  Special  Project*  Office 
BUORD 


!/\>»AaCfAY 
Bnpdier  General,  USA 
Commander,  ABMA  / 


Figure  4 


MANAGEMENT  POLICIES  FOR  ASSIGNING  DEPARTMENTAL 
RELIABILITY  RESPONSIBILITIES 
by  Leslie  W.  Ball 

Management  Consultant-Product  Reliability!  El  Cajon,  California 
STATUS  OF  RELIABILITY  ENGINEERING 


About  ten  years  ago,  reliability  engineering  entered  its  first  phase. 

The  name  "conceptual"  would  be  appropriate  to  this  phase  because  at 
that  time  the  greatest  need  was  to  formulate  and  propagate  concepts 
such  as  the  Product  Rule.  After  the  efforts  of  the  pioneers,  such  as 
Robert  Lusser,  had  drawn  attention  to  the  problem  and  events  in 
Korea  had  emphasized  its  importanceyreliability  engineering  entered 
the  "fact  finding"  phase.  Service  failure  data  gathered  by  AIRinc. 
Research  Corporation  and  others  confirmed  the  pessimistic  opinions 
of  a  growing  band  of  reliabiltlity  specialists.  It  was  recognised  that 
new  methods  were  required  to  deal  with  the  so  called  "early"  and 
"random"  types  of  failures.  Consequently,  reliability  engineering 
entered  the  "technical  methods  research"  phase. 

In  this  year  I960  conceptual,  fact  finding  and  technical  methods  re¬ 
search  efforts  continue  to  be  important  but  technical  knowledge  has 
advanced  to  a  point  where  the  most  urgent  need  is  now  for  "manage¬ 
ment  methods  research".  This  is  not  an  academic  opinion.  Its 
validity  is  being  demonstrated  by  current  BMC /STL  experience  on  the 
Minuteman  Intercontinental  Ballistic  Missile  Program.  In  this  program 
achievement  of  overall  weapons  system  cost  effectiveness  through 
reliability  improvement  is  the  principle  objective.  We  must  and  will 
achieve  two  orders  of  magnitude  improvement  in  the  failure  rates  of 
electronic  and  electro -mechanical  parts,  but  to  do  so  it  has  become 
necessary  to  concentrate  our  reliability  efforts  in  the  area  of  manage¬ 
ment  methods  research  in  general  and  parts  supplier  control  in 
particular. 

In  brief,  we  may  say  that  in  the  year  I960  retention  of  our  military 
superiority  depends  upon  achieving  superior  reliability  in  complex 
weapon  systems  and  that  in  turn  this  superiority  depends  upon  the 
ability  and  dedication  with  which  we  attack  the  problem  of  reliability 
management  methods  research. 
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UNDERLYING  CAUSES  OF  UNRELIABILITY 


Mere  concepts  or  even  undigested  facts  are  not  a  satisfactory  base  for 
management  action.  It  is  necessary  to  work  out  a  simple  but  funda¬ 
mental  understanding  of  the  underlying  causes  of  unreliability  and  to 
express  this  understanefing  in  management  terms. 

The  authors  choice  of  underlying  causes  is  based  on  failure  analysis. 
The  results  of  each  analysis  have  been  expressed  in  tabular  form 
using  the  following  four  column  titles:  (1)  Item.  (2)  Failure  Phenom¬ 
enon,  (3)  Controllable  Human  Error  and  (4)  Recurrence  Control  Action. 

The  first  column  names  the  part  that  failed.  This  is  justified  because 
every  failure  is  the  failure  of  a  part  even  though  the  system  produces 
the  stresses  and  performance  requirement  that  cause  failures. 

The  second  column  gives  a  description  in  engineering  terms  of  what 
actually  happened.  ^ 

The  third  column  "Controllable  Human  Error"  is  absolutely  essential 
to  paving  the  way  for  management  action  to  prevent  recurrence, 
hneause  management  can  not  control  the  law  of  physics  but  can  con¬ 
trol  the  actions  oi  their  employees. 

The  fourth  column  "Recurrence  Control  Action"  provides  an  answer  to 
the  question,  "What  specific-discipline  or  practice  could  management 
require  and  enforce  to  reduce  the  probability  of  recurrence  of  this 
same  failure?  ". 

Failure  analyses  performed  in  accordance  with  the  above  pattern  have 
led  to  the  conclusion  that  »he  underlying  causes  of  unreliability  are: 

(1)  Variance  in  Materials  and  Processes.  Customer  demands  for 
extreme  performance  combined  with  a  national  tendency  to  grab 
the  novel  have  led  to  the  use  of  materials  and  processes  that  have 
associated  with  them  variances  that  cause  myriads  of  infrequently 
occurring  modes  of  failure. 

(2)  Lack  of  Organized  Knowledge.  Organized  knowledge  of  "normal" 
modes  of  failure  is  available  to  design  and  mwufacturing  engineers 
in  handbook  form  but  knowledge  required  for  the  prediction,  con¬ 
trol  and  measurement  of  infrequently  occurring  modes  of  failure 
has  not  been  adequate. 

(3)  Bre  akdown  of  Management  Discipline,  In  structural  engineering 
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absolute  reliability  has  been  achieved  by  management  enforce* 
ment  of  the  discipline  of  stress  analysis,  qualification  testing, 
process  control  and  conformance  inspection.  In  the  missile 
electronics  and  even  in  the  missile  mechanical  industry  failure 
analyses  show  that  these  disciplines  have  deteriorated.  For 
example,  in  the  case  of  electronics  the  discipline  of  stress 
analysis  was  hardly  recognised  until  a  few  years  ago. 

Because  it  is  not  possible  to  achieve  the  performance  required  of 
missiles  without  the  use  of  materials  and  processes  that  carry  with 
them  myriads  of  infrequently  occurring  modes  cf  failure  it  is  not 
possible  to  restore  absolute  reliability.  However,  it  is  possible  to 
provide  through  reliability  research  the  data  and  methods  required  to 
predict  and  control  failure  rates  and  to  provide  reliability  assurance 
methods  for  ensuring  that  the  disciplines  necessary  to  controlling 
failure  rates  are  followed. 

GENERAL  RELIABILITY  POLICY  STATEMENT 

It  is  totally  unsatisfactory  for  the  general  manager  of  a  missile 
company  to  meet  a  customer  demand  for  a  reliability  program  merely 
by  creating  a  reliability  engineering  group.  Reliability  achievement 
requires  a  positive  series  of  actions  at  all  levels  of  the  organization. 
This  series  of  actions  must  start  at  the  top  and  be  generated  down 
with  the  full  weight  of  line  organisation  authority. 

A  general  manager  can  discharge  his  responsibility  by  issuing  a 
"General  Reliability  Policy  Statement"  provided  that  this  statement  is 
specific  and  comprehensive  enough  to  cover  (1)  Purpose,  (2)  Organiz¬ 
ation,  (3)  Practices,  (4)  Personnel  and  (5)  Audit.  Each  of  these  items 
will  be  discussed  in  the  following  sections  of  this  paper. 

RELIABILITY  POLICY  STATEMENT -PURPOSE 

The  general  reliability  policy  statement  of  purpose  should  be  derived 
from  the  above  list  of  underlying  causes.  The  following  is  a  model 
statement: 

I.  PURPOSE 

The  John  Doe  Company  will  maintain  a  company-wide  reliability 
program.  The  purpose  of  this  program  is  to  attain  maximum 
systems  effectiveness  within  available  funds  by  providing  organ¬ 
ized  knowledge  and  by  enforcing  effectiveness  disciplines  for  pre¬ 
diction,  control,  measurement,  and  reporting  of  both  normal  and 
Infrequently  occurring  modes  of  failure. 
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RELIABILITY  POLICY  STATEMENT-ORGANIZATION 


The  policy  statement  on  organization  should  not  deal  with  organization 
chart  details.  Such  details  may  result  from  expediency  and  be  subject 
to  change.  It  should  deal  with  those  aspects  of  organization  that  are 
vitally  important  to  reliability  achievement  and  fundamental  enough  to 
have  long  term  stability. 

The  question  "Who  is  responsible  for  reliability?"  does  not  lend  itself 
to  a  helpful  answer  because  reliability  activities  are  so  widespread 
throughout  an  organization  that  only  the  general  manager  can  be  re¬ 
sponsible  for  all  aspects. 

Much  more -useful  answers  can  be  generated  by  asking  four  questions. 
"Who  is  responsible  for  reliability  research,  who  is  responsible  for 
reliability  creation,  who  is  responsible  for  reliability  assurance  and 
who  is  responsible  for  project  reliability  management?".  The  follow¬ 
ing  model  general  reliability  policy  statement  follows  this  pattern. 

It.  ORGANIZATION 


(a)  Reliability  Creation.  The  design  departments  are  and  will 
continue  to  be  responsible  for  building  reliability  into  their  designs 
The  purchasing  and  manufacturing  departments  are  and  will  con¬ 
tinue  to  be  responsible  for  building  reliability  Into  production 
hardware. 

(b)  Reliability  Research  and  Assurance.  A  reliability  director 
reporting  directly  to  the  general  manager  will  be  responsible  for 
ensuring  that  the  knowledge,  practices  and  skills  required  for 
reliability  creation  and  reliability  assurance  are  developed  and 
made  available  as  needed  throughout  the  organization. 

(c)  Reliability  Assurance.  Design  reliability  assurance  and  pro 
duction  reliability  assurance  groups  will  be  established  either 
within  the  engineering  and  quality  departments  or  reporting 
directly  to  the  reliability  director. 

(d)  Project  Reliability  Management.  For  each  project  for  which  a 
reliability  program  is  established  a  reliability  project  manager 
will  be  appointed  to  ensure  the  coordination  and  effectiveness  of 
all  aspects  of  the  project  reliability  program. 

It  should  be  noted  that  the  above  statement  of  policy  permits  but  does 
not  enforce  gathering  all  the  reliability  research  and  assurance  groups 
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together  into  one  line  organisation  under  a  reliability  director.  The 
essential  organization  requirements  are  (1)  that  these  groups  be 
established  and  (2)  that  the  reliability  director  have  centralized  function¬ 
al  control  over  them.  Decentralized  location  and  line  administration 
are  not  incompatible  with  centralized  functional  responsibility  and 
control.  Table  I.  illustrates  this  organizational  principle. 

•  RELIABILITY  POLICY  STATEMENT-PRACTICES 

The  reliability  program  must  control  the  detailed  activities  of  every 
drawing  board  designer,  buyer,  manufacturing  floor  or  bench  worker, 

«  inspector  and  service  support  technician.  There  are  two  major  aspects 

to  accomplishing  this  requirement. 

First,  every  activity  that  affects  equipment  reliability  must  be  ident¬ 
ified  and  within  each  activity  required  procedures  must  be  catalogued. 
Table  II.  presents  a  list  of  thirty  activities.  This  list  has  evolved 
from  the  failure  analysis  process  in  answer  to  the  question  "What 
specific  activity  or  discipline  could  management  require  to  prevent 
recurrence  of  this  failure  in  the  future?".  In  addition,  every  Depart¬ 
ment  of  Defense  specification,  policy  or  contractual  requirement  for 
reliability  has  been  checked  to  ensure  that  the  list  of  activities  was 
complete.  Of  course.  Table  II.  shows  only  brief  titles  that  are  subject 
to  misinterpretation  as  to  meaning  and  scope^  The  author  has  com¬ 
pleted  an  "Activities  Policy  Manual"  which  includes  a  model  policy 
statement  for  each  activity  and  a  list  of  applicable  principles  that  are 
so  fundamental  that  they  apply  to  all  types  of  organization  and  product. 

A  book  now  under  preparation  further  elucidates  the  nature  and  scope 
of  each  of  these  activities. 

The  second  requirement  is  that  management  discipline  within  the 
company  must  provide  for  a  continuum  of  documented  authority  from 
the  general  reliability  policy  statement  down  to  the  actual  operating 
instructions  used  by  the  bench  worker.  Both  the  requirement  for 
identification  of  activities  and  the  continuum  of  authority  are  covered 
in  the  following  model  policy  statement. 

Ill.  PRACTICES 

% 

A  continuum  of  management  control  over  reliability  practices  will 
be  established  and  maintained  by  the  following  sequence  of  author¬ 
itative  documents: 

4 

(a)  Reliability  Policies  Manual.  The  Reliability  Director  will 
issue  and  maintain  a  Reliability  Policies  Manual  that  will. 
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(1)  identify  specific  activities  essential  to  achieving  the  above 
purpose,  (2)  provide  policy  guidance  for  each  activity.  (3)  define 
departmental  responsibilites  for  each  activity. 

(b)  Reliability  Procedure  Guides.  The  Reliability  Director  will 
identify  all  the  procedures  that  must  be  written  by  each  functional 
department  to  implement  the  Policies  Manual  and  will  provide  a 
guide  for  each  such  procedure. 

(c)  Department  Procedures  and  Instructions.  Each  functional 
department  will  issue  Department  Procedures  as  required  to 
implement  the  Reliability  Policies  Manual  and  such  further  Oper- 
ating  Instructions  as  are  required  to  control  each  practice. 

RELIABILITY  POLICY  STATEMENT -PERSONNEL 

During  the  phase  in  which  the  principle  job  of  the  reliability  engineer 
was  to  spread  concepts  and  to  gain  attention  to  reliability  from  his 
colleagues,  reliability  groups  could  be  staffed  by  "generalists".  Now 
that  reliability  program  implementation  requires  very  specific  activit¬ 
ies,  staffing  by  generalists  is  proving  to  be  chronically  ineffective. 

The  time  as  come  when  extremely  serious  attention  must  be  paid  to  the 
specific  skills  and  training  required  to  make  reliability  engineering 
personnel  effective.  An  essential  aspect  of  this  study  must  be  to  pro¬ 
vide  long  term  career  growth  possibilities  for  motivation  of  on-board 
personnel  and  for  recruiting  oi  new  talent. 

Table  1.  lists  eleven  types  of  reliability  engineering  skill  and  corr¬ 
elates  these  skills  with  the  reliability  activities  list.  Again,  the  brief 
titles  used  for  the  skill  groups  are  subject  to  misinterpretation  unless 
model  position  descriptions  are  provided  for  each.  The  scope  of  this 
paper  is  limited  to  emphasizing  the  extreme  importance  of  having  the 
skill  groups  well  defined  ratner  than  providing  the  position  descript¬ 
ions. 

A  model  general  reliability  policy  statement  on  personnel  is  as  follows. 
IV.  PERSONNEL 

The  specialized  skills  required  for  implementation  of  reliability 
practices  will  be  identified  and  job  categories  established.  The 
Reliability  Director  will  ensure  the  professional  adequacy  of 
these  skill  groups. 


RELIABILITY  POLIC  S  ~AT  EMENT  -  AUDIT 


It  is  extremely  important  to  management  understanding  to  recognise 
that  for  procurement  of  high  reliability  parts  and  equipment  a  tradition¬ 
al  ;  lationship  between  buyer  and  seller  has  broken  down.  Tradition¬ 
ally,  the  buyer  would  write  a  specification  describing  the  performance 
characteristics  that  had  to  be  met  and  the  buyer  and  seller  would  agree 
on  a  test  program  that  would  be  satisfactory  for  demonstration  of 
contract  fulfillment.  For  procurement  of  high  reliability  the  principle 
of  specifying  a  quantitati  e  value  is  fundamental  but  its  application  is 
not  simple.  It  is  complicated  partly  by  the  fact  that  reliability  is  a 
growth  characteristic  and  can  best  be  represented  by  a  curve  relating 
reliability  to  calendar  time.  The  initial  reliability  of  the  prototype  is 
only  one  point  on  this  curve.  It  is  complicated  further  by  the  fact  that, 
with  the  exception  of  electronic  equipment  for  which  the  requirement  is 
a  mean  time  between  failure  of  less  than  100  hours,  demonstration  by 
testing  alone  is  prohibitively  expensive,  time  consuming  and  inconclus¬ 
ive.  Even  if  reliability  could  be  demonstrated  by  testing  with  re  ison- 
able  cost  and  accuracy  this  demonstration  often  would  come  so  late  in 
a  program  that  by  the  time  that  failure  to  meet  the  reliability  require¬ 
ment  was  recognised  the  loss  of  lead  .ime  would  be  catastrophic. 

Because  the  buyer  cannot  contract  for  reliability  simply  by  require¬ 
ments  for  testing  the  sellers  output,  it  is  absolutely  essential  to  con¬ 
tract  for  control  and  visibility  of  the  reliability  that  the  seller  builds 
into  his  design,  purchasing  and  manufacturing  work.  It  is  to  be  ex¬ 
pected  that  the  sellers  first  reaction  to  demands  for  extreme  visibility 
of  his  internal  operations  will  be  resentment.  This  resentment  will  be 
increased  if  the  buyer  is  not  clear  on  just  what  type  of  visibility  is 
needed  and  proposes  to  use  an  excessive  number  of  resident  buyer 
engineers . 

This  type  of  problem  has  been  a  major  factor  in  BMC/STL  development 
of  management  methods  for  the  Minuteman  Reliability  Program.  It  has 
been  found  helpful  to  convince  suppliers  that  every  scrap  of  information 
that  the  buyer  is  seeking  is  essential  to  their, own  management  control 
over  their  reliability  programs. 

Attempts  to  achieve  both  management  visibility  and  project  control  by 
means  of  a  single  "Project  Reliability  Program  Report"  covering  both 
the  contractors  basic  practices  and  the  particular  project  have  proved 
to  be  unsatisfactory.  Major  improvements  are  being  gained  by  re¬ 
quiring  a  "Company  Basic  Program  Report"  and  a  separate  "Project 
prpgram  Plan".  The  former  document  summarizes  all  the  policies, 
practices  and  experience  of  the  company  that  are  available  for 
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application  to  any  project.  The  second  document  then  can  be  re¬ 
stricted  to  a  catalogue  of  specific  tasks  for  each  of  which  there  is  a 
definite  objective,  fi  d  allocation  and  time  phasing. 

The  following  is  a  i  o..el  fjeneral  reliability  policy  statemert  on  audit: 

V.  AUDIT 

Management  visibility  of  the  status  of  reliability  practices,  skills 
and  project  programs  will  be  provided  by: 

(a)  Basic  Reliability  Program  Report.  The  Reliability  Director 


will  issue  and  maintain  a  Basic  Reliability  Program  Report.  This 
document  will  summarize  the  current  status  of  the  policies,  pro¬ 
cedures,  instruction  and  skill  groups  that  have  been  developed  for 
implementing  any  project  reliability  program. 

(b)  Project  Reliability  Program  PI  ans.  Each  customer  require¬ 


ment  for  a  Project  Reliaoility  Program  will  be  met  by  establisn- 
ment  and  execution  of  a  Project  F.eliability  Program  Plan.  Each 
such  plan  will  consist  of  clearly  defined  tasks  prepared  in  accord¬ 
ance  with  the  Reliability  Policy  Manual  and  the  Basic  Reliability 
Program  Report.  Progress  reports  on  each  task  will  be  specific 
and  will  be  distributed  to  project  managers,  company  management 
and  to  the  customer. 

SUMMARY 

In  this  year  l?uO,  the  most  urgent  aspect  of  reliability  engineering  is 
the  need  for  management  methods  research  and  for  implementation  of 
methods  already  developed.  Enough  work  has  been  done  for  the  re¬ 
liability  engineering  profession  to  offer  to  management  a  clear  path 
for  constructive  action.  General  management  can  fulfill  their  re¬ 
sponsibilities  and  provide  effective  delegation  to  the  next  organization¬ 
al  level  by  issuing  a  General  Reliability  Policy  Statement  covering 
Purpose,  Organization,  Practices,  Personnel  and  Audit  and  by  backing 
implementation  of  the  policy  statement  after  it  has  been  issuer1.  This 
symposium  can  contribute  to  improved  management  of  reliability  pro¬ 
grams  by  spreading  unde  r s tandi n g  among  the  engineering  attendees 
and  by  clarifying  a  course  of  action  for  more  effective  contract  re¬ 
quirements  among  the  Department  of  Defense  attendees. 
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Introduction 

ihe  task  of  designing  and  building  reliable  electronic  equipment  is 
rapidly  becoming  one  of  the  most  important  problems  in  the  electronics 
industry.  Reliability  has  been  brought  to  the  attention  of  everyone 
in  a  very  vivid  manner  during  the  last  two  years  by  some  spectacular 
failures  in  attempts  to  launch  satellites  and  missiles.  The  Defense 
Department  is  now  writing  reliability  requirements  into  all  contracts 
for  weapons  systems.  These  requirements  are  already  having  reper¬ 
cussions  throughout  the  whole  electronics  industry  from  component 
manufacturers  and  their  suppliers  to  system  designers  and  fabricators. 
The  impact  will  be  felt  increasingly  within  the  next  few  years. 


Reliability  problems  are  not  restricted  to  the  manufacture  of  military 
systems  alone.  The  importance  of  reliability  for  industrial  applications 
i6  becoming  increasingly  epparent  as  more  and  more  large  scale  computers 
and  other  complex  electronic  equipment  are  being  employed  by  Industry. 

The  problems  will  become  increasingly  acute  as  automation  with  its 
myriads  of  components  and  equipment  comes  into  more  widespread  use. 

Inasmuch  as  reliability  has  come  to  be  a  major  problem,  it  is  desirable 
to  examine  the  whole  subject.  What  is  reliability?  What  makes  the 
problem  so  difficult?  What  is  being  done  and  what  can  be  done  to  design 
and  build  reliable  equipment?  We  shall  investigate  these  points  very 
briefly  in  what  follows. 


Reliability 


An  Old  Problem  in  New  Form 


In  order  to  discuss  reliability,  it  is  first  necessary  to  define  vhat  is 
meant  by  the  term.  Basically  we  mean  that  whatever  we  are  considering  is 
reliable  if  it  almost  always  works  when  we  want  it  to.  The  terra  almost 
always  sugcests  that  reliability  is  somehow  associated  with  probability. 
That  is  correct.  Ve  can  never  be  1005>  certain  that  something  will  work 
when  we  want  it  to.  However,  it  may  be  possible  to  build  a  system  which 
is  likely  to  work  almost  100 i  of  the  time.  We  say,  then,  the  system  has 
a  probability  of  .9999  or  -95  or  some  other  positive  number  less  than  one. 
It  is  sometimes  convenient  to  define  reliability  as  the  probability  that 
whatever  we  arc  dircucrin;*  will  operate  in  a  satisfactory  way  when  we  want 
it  to. 
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Time  usually  enters  into  discussions  of  reliability.  It  is  not 
sufficient  that  everything  works  in  a  satisfactory  way  at  the  instant 
a  missile  is  launched.  The  equipment  must  continue  to  operate  for  a 
length  of  time  necessary  to  assure  the  success  of  the  launching,  however 
long  this  may  be.  Thus,  when  one  states  reliability  requirements,  time 
often  appears  in  the  specification,  i.e.,  with  probability  .98  a  computer 
must  operate  without  a  failure  for  100  hours.  The  .98  is  called  the 
reliability  index  of  the  system. 

The  reliability  problem  is  not  a  new  one.  It  has  been  met  and  BOlved  with 
some  success  in  a  large  number  of  industries.  For  example,  the  manu¬ 
facturers  of  light  bulbs,  electron  tubes,  and  automobile  batteries  have 
been  concerned  with  turning  out  reliable  parts  for  many  years.  The  task 
facing  the  electronics  industry  is  of  a  different  order  of  magnitude, 
however.  Many  electronic  systems  have  30,000  or  more  electronic  components 
ir  them.  The  task  of  getting  all  these  parts  to  perform  properly  is  quite 
different  from  that  needed  to  be  reasonably  sure  that  a  single  light  bulb 
will  operate.  In  addition,  the  reliability  requirements  are  often  much 
higher  on  these  complex  systems  than  they  are  for,  say,  automobile 
batteries.  If  a  missile  foils  on  launching,  several  million  dollars  are 
lost.  Even  worse,  national  security  is  dependent  in  part  on  reliable 
systems.  When  a  car  battery  fails  it  is  only  necessary  to  replace  the 
battery.  The  only  unfortunate  result  may  be  a  dissatisfied  customer. 

At  the  present  time,  the  reliability  requirements  for  very  complex 
electronic  systems  axe  often  for  very  short  times,  of  the  order  of  hours 
or  les6.  These  have  been  very  difficult  to  meet.  However,  the  trends 
both  for  military  and  civilian  applications  are  toward  high  reliability 
requirements  for  very  long  operating  times  without  maintenance.  For 
example,  a  successful  unmanned  flight  to  Venus  Or  Mars  would  require  the 
electronic  equipment  to  operate  for  thousands  of  hours  without  maintenance 
and  without  a  failure.  Similarly  it  is  becoming  desirable  to  have 
missiles  sit  in  readiness  for  firing  for  years  with  little  or  no 
maintenance.  Equally  well  it  is  becoming  a  requirement  that  large  scale 
digital  computers  in  industry  operate  for  long  periods  without  failures. 

The  same  requirements  will  have  to  be  met  by  the  electronic  equipment  in 
automated  factories  - 

It  should  be  clear  that  the  task  of  making  a  very  complex  piece  of 
electronic  equipment  operate  for  long  periods  without  failure  Is  not  an 
easy  one.  A  program  of  major  proportions  will  be  needed  in  the  entire 
electronics  industry  to  produce  such  systems. 

Facets  of  Reliability 


The  manufacture  of  complex  electronic  systems  which  meet  certain  high 
reliability  requirements  is  what  might  tritely  be  called  a  cradle  to  the 
grave  operation.  The  responsibility  extends  far  beyond  the  company  which 
designs  and  builds  the  system-  It  begins  with  the  component  manufacturers 
and  the  Gupplierr  of  the  component  manufacturers.  Reliable  systems 
require  reliable  components  and  components  designed  to  meet  the  specific 
reliability  requirements  of  the  system  as  well  as  to  meet  performance 
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requirements.  Next,  reliability  must  be  part  of  the  design  specifications. 
Circuits  must  be  designed  vith  reliability  as. one  of  the  most  important 
requirements  to  be  met.  Of  course,  the  manufacturer's  process  and 
manufacturing  techniques  must  also  be  geared  to  the  production  of  reliable 
systems.  The  emphasis  on  reliability  does  not  even  cease  vhen  the  systems 
have  been  produced.  It  is  necessary  to  record  the  behavior  of  these 
systems  during  actual  operation  in  the  field.--  When  failures  do  occur, 
careful  analysis  must  be  made  to  ascertain  vhat  caused  the  failure.  Was 
it  a  poor  component,  faulty  design,  or  vhat?  This  information  must  then 
be  fed  back  to  the  proper  departments  as  an  aid  in  improving  the 
reliability  of  future  systems. 

The  management  task  of  assuring  the  production . of  reliable  systems 
requires  special  organizational  structures  vhich  are  seldom  found  in 
present  day  organizations.  Reliability  requirements  extend  beyond 
engineering  only,  manufacturing  only,  or  purchasing  only.  The  organiza¬ 
tional  structure  must  be  such  that  the  reliability  activities  in  all 
areas  can  be  coordinated  so  that  all  work  toward  the  final  objective 
rather  than  having  each  department  work  toward  its  own  objectives  vhich 
may  be  incompatible  with  producing  reliable  systems. 

The  technical  and  administrative  aspects  of  producing  reliable  systems 
will  next  be  investigated.  As  we  shall  see,  there  are  a  number  of  areas 
in  which  techniques  are  not  now  available. 

TECHNICAL  ASPECTS  OF  RELIABILITY 

Reliability  Apportionment 

Reliability  requirements  for  complete  systems  are  now  being  specified  in 
military  contracts.  For  industrial  applications  there  will  be  certain 
minimum  reliability  requirements  if  the  system  is  to  be  feasible.  The 
determination  of  the  actual  reliability  specification  in  such  a  case 
requires  a  trade-off  study  between  cost,  performance,  reliability,  and 
customer  satisfaction. 


Assuming  that  a  reliability  requirement  1b  given  for  the  entire  system, 
it  then  becomes  necessary  to  break  down  this  requirement  into  reliability 
requirements  for  subsystems,  then  circuits,  and  also  conqjonents.  In 
other  words,  it  is  necessary  to  break  down  the  over-all  reliability 
requirement  into  something  which  circuit  designers  can  use  and  something 
vhich  can  be  used  to  write  specifications  for  component  parts. 

At  the  present  time,  there  is  no  general  procedure  available  for  making 
such  a  reliability  breakdown.  The  reliability  breakdown  must  be  such 
that  in  addition  to  the  reliability  requirements,  restrictions  on  weight, 
volume,  etc.,  as  well  as  the  performance  characteristics,  can  be  met. 

In  order  to  talk  about  making  an  optimal  breakdown  of  the  over-all 
reliability  requirement  one  must,  have  some  objective  --  such  as  minimizing 
the  total  system  cost.  The  breakdown  cannot  really  be  made  optimally 
independent  of  the  circuit  design  since  the  design  has  a  marked  influence 
on  reliability.  In  reality  the  circuit  design  should  be  established 
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simultaneously  with  the  reliability  allocation  if  a  true  optimal  solution 
is  desired.  This  would  include  selection  of  the  proper  components  to  be 
used  in  the  circuit.  Here  one  encounters  additional  difficulties  because 
the  components  will  often  need  to  be  designed  specifically  to  meet  the 
reliability  requirements.  Any  general  model  for  opportionlng  the 
reliability  must  then  also  consider  the  probabilities  and  costs  for 
developing  components. 

The  above  discussion  points  out  that  the  development  of  a  model  for 
optimal  apportionment  of  the  system  reliability  appears  to  be  difficult 
to  develop.  A  great  deal  of  work  must  be  devoted  to  finding  good  ways 
to  break  down  the  system  reliability  requirement  to  the  circuit  and 
component  levels.  It  is  not  exactly  clear  what  analytical  tools  are  in 
use  today  for  this.  In  reality  it  appears  that  most  reliability  break¬ 
downs  are  based  more  upon  intuitive  judgment  rather  than  upon  analytical 
methods.  Whether  this  is  good  or  bad  depends  upon  how  good  is  the 
intuitive  Judgment  of  the  individual  making  the  breakdown. 

The  Sacred  Exponential  Distribution 

Most  of  the  statistics  associated  with  reliability  computations  are  based 
upon  what  is  known  as  the  exponential  fad. lure  distribution.  If  P(t)  is 
the  probability  that  a  system  or  component  which  began  operation  at  time 
zero  survives  up  to  time  t,  then  according  to  the  exponential  distribution 


P(t)  =  e-Xl  (1) 

where  Ue  a  constant  independent  of  time,  called  the  failure  rate.  The 
probability  that  the  system  will  have  failed  at  some  time  between  0  and  t 
is  then 

Q(t)  -  l-P(t)  =  l-e_Xt  (2) 

This  is  the  cumulative  probability  of  failure.  The  curve  is  shown  in 
Figure  1. 

Differentiation  gives  the  failure  density  function  f(t) 


f(t)  =  Xe"Xt  (3) 

The  apriori  probability  that  the  system  will  fail  between  time  t  and  t«-dt 
Is  f(t)dt.  The  mean  of  this  distribution  is 

oc 

t.  =  \J0  te_Xt  dt  =  1/ A  (U) 
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Consequently  1/  the  reciprocal  of  the  failure  rate,  is  called  the  mean 
time  to  failure.  When  t  =  1  / X  ye  see  from  (l)  the  probability  that  the 
system  is  still  operating  is  e"1  =  .36  (6ee  Figure  1). 


Figure  1.  Cumulative  Probability  of  Failure 

It)  compute  the  probability  that  the  By b tern  will  fail  in  the  interval 
t  to  t  +  dt  given  that  it  has  survived  to  time  t,  we  use  the  general 
conditional  probability  formula  (assuming  A  and  B  independent; 

P(A)  =  p(a|b)  P(B).  (5) 

Where  P(A)  i6  the  apriori  probability  of  event  A,  P(a|b)  is  the 
conditional  probability  of  A  given  that  the  event  B  has  occurred; 
and  P(B)  is  the  apriori  probability  of  event  B;  but 

P(A)  =  f(t)dt  =  Ae'Xt  dt;  P(b)  =  e"Xt 

Thus  the  probability  that  system  foils  between  time  t  and  t  ♦  dt, 

P(A|B),  given  that  it  has  survived  to  time  t  is  Mt.  This  Is  a  very 
interesting  result  because  it  says  that  the  probability  that  a  failure  will 
occur  in  the  time  interval  t  to  t  ♦  dt  is  independent  of  how  long  the 
system  has  been  operating.  This  also  explains  why  is  referred  to  as  a 
failure  rate. 

The  exponential  distribution  has  Dome  other  very  convenient  properties 
in  addition  to  those  mentioned  above.  Suppose  we  imagine  a  system  be 
formed  from  a  number  of  parts,  enen  of  which  obeys  the  exponential 
failure  rate  distribution.  Let  the  failure  rate  of  part  i  be  Aj  . 

Tnen  if  a  faille  of  any  on<r  componenrt -1  s  independent  of  the  others, 
the  probability  P(T)  that  the  system  will  survive  from  time  0  to  t  is 
the  product  of  the  probabilities  that  each  component  will  survive. 

Tnus  if  there  arc  n  components 
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P{t)  -  {exp  -Xjt)(exp  *  X  ^  t ) 


(exp_-Xnt)»  exp 

(6) 

Equation  (6)  shovB  that  the  system  also  obeys  an  exponential  failure 
rate  law,  and  the  failure  rate  X  for  the  whole  system  is  Just 


All  these  properties  of  the  exponential  distribution  indicate  that  it 
is  quite  easy  to  make  reliability  computations  if  the  assumption  is 
made  that  everything  obeys  the  exponential  failure  rate  lav. 

Although  the  exponential  distribution  has  very  desirable  mathematical 
properties,  the  real  question  which  must  be  answered  is  hov  veil  does 
it  represent  failures  of  components  and  9yi  terns.  The  answer  to  this 
question  does  not  seem  to  be  available  toiay.  Although  there  is  some 
evidence  that  failures  of  certain  components  obey  the  exponential 
distribution,  it  is  not  desur  (and  almost  certainly  not  true)  that  all 
components  do.  Even  less  can  be  said  about  the  distributions  of 
failures  for  complex  systems.  It  is  quite  important  to  learn  something 
about  these  distributions  because  realistic  reliability  computations 
cannot  be  made  without  some  knowledge  of  these  matters. 

It  is  worthwhile  to  keep  in  mind  setae  properties  of  the  exponential 
distribution.  First  it  can  be  noted  that  it  does  not  account  for  wear 
out.  If  a  system  has  survived  up  to  time  t,  the  probability  that  it 
will  fail  in  the  next  kime  interval  is  independent  of  how  long  the 
system  has  been  in  op -ration.  The  probability  is  the  sane  lamellate ly 
after  the  system  has  been  started  as  it  is  after  the  system  has  been 
operating  for  1000  hours.  Ultimately  any  Bystem  will  wear  out.  The 
exponential  distribution  also  ignores  particular  failure  mechanisms. 
Essentially  what  it  assumes  is  that  there  are  a  very  large  number  of 
ways  in  which  a  system  can  fail.  The  number  is  so  large  that  failures 
can  be .considered  to  occur  at  random. 

The  specification  of  a  mean  time  to  failure  can  be  very  misleading  if 
one  is  not  careful.  For  example,  a  system  which  is  to  operate  for  one 
year  with  high  reliability  may  require  a  me  am  time  to  failure  of  perhaps 
25  years  if  one  uses  the  exponential  distribution  to  perform  the 
computations.  There  is  no  implication  here  that  tne  system  must  laa.t 
25  years.  It  may  be  worn  out  completely  in  a  year  and  one  day.  Ho '/ever, 
if  the  systems  obey  the  exponential  failure  law  during  the  one  year  of 
operation,  this  distribution  must  have  l/x  *  25  years.  The  mean  time  to 
failure  i6  the  only  parameter  in  the  exponential  distribution,  and  hence 
when  X  is  fixed,  the  distribution  is  determined. 

For  the  system  discussed  above,  it  may  be  necessary  for  one  of  the 
components  going  into  the  system  to  have  a  mean  time  to  failure  of 
100,000  years.  Clearly  thiB  number  has  nothing  to  do  with  the  length 
of  time  over  which  the  component  must  operate.  It  is  only  necessary 


that  the  component  operate  for  one  year  in  the  system.  Hie  value 
1/X  =  100,000  years  determines  the  exponential  distribution  for  this 
type  of  component  and  thus  determines  the  probability  that  a  component 
will  fail  within  a  year. 

£•- liability  and  Circuit  Design 

One  of  the  most  important  ways  of  improving  reliability  is  through 
circuit  design.  Unfortunately,  however,  very  few  techniques  have  been 
developed  to  aid  the  designer  to  devise  reliable  circuits.  Part  of 
the  task  is  to  have  circuits  which  will  not  overload  the  components 
under  normal  operating  conditions,  thus  causing  a  failure.  This 
problem  appears  to  be  reasonably  straight  forward,  involving  little 
more  than  using  suitable  deratings  on  components.  Things  are  not 
Always  quite  this  simple  in  practice  either  because  performance  cannot 
be  sacrificed  cr  because  suitable  components  are  not  available. 

To  decrease  the  likelihood  of  system  fad  lure  due  to  &  complete  collapse 
of  some  component  for  reasons  which  lie  outside  the  circuit,  it  has 
often  been  suggested  that  redundancy  in  components  be  used.  This  has 
not  been  found  to  be  too  helpful  in  many  cases  because  the  complexity  of 
the  switching  circuits  reduces  the  reliability  to  a  point  where  little 
is  gained  from  redundancy  and  quite  a  bit  is  lost  in  the  parameters  of 
cost,  weight,  and  volume. 

Often  a  system  fails,  not  because  a  component  has  failed  completely, 
but  rather  because  some  characteristic  of  the  component  has  drifted  with 
time  outside  of  the  tolerances  which  allow  the  circuit  to  perform  in  a 
satisfactory  way.  In  order  to  know  if  a  circuit  1b  reliable,  the 
designer  must  know  how  sensitive  the  circuit  is  to  drift,  because  nearly 
all  components  will  drift  with  time.  Essentially  no  techniques  are  ( 
available  at  present  for  performing  such  analyses  although  work  is 
underway  to  develop  them.  The  problem  is  very  complicated  if  all  parts 
are  allowed  to  drift  simultaneously.  However,  it  i6  quite  likely  that 
sufficient  information  can  be  obtained  by  allowing  a  small  number  of 
parameters  to  vary  simultaneously. 

Ir.  addition  to  the  type  of  circuit  analysis  described  above,  it  is  also 
of  interest  to  be  able  to  have  some  way  of  computing  the  possible 
behavior  of  systems  constructed  using  components  having  known  probability 
distributions  for  the  nominal  values  of  the  parameter  and  known  distri¬ 
butions  for  drift  as  a  function  of  time.  In  other  words,  if  components 
are  chosen  at  random  from  inventory  to  build  a  system,  what  sort  of 
variations  can  be  expected  in  the  system  performance  due  to  normal 
variations  in  the  components.  This  problem  is  important  from  the 
reliability  point  of  view  and. for  the  associated  problems  involved  with 
interchanging  and  mating  subsystems .  Monte  Carlo  analysis  will  quite 
likely  be  a  very  useful  technique  in  studying  this  type  of  problem. 
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Individual  Component  Reliability  ^ 

The  reliability  of  a  complex  system  depends  strongly  on  the  use  of 
extremely  reliable  components.  '-  We  must  begin  by  asking  vhat  a  reliable 
c'er-'nent  is.  A  reliable  component  1b  one  vhich  will  operate  and  have 
all  pertinent  parameters  remain  vithin  tolerance  for  a  long  enough 
time  and  with  a  high  enough  probability  that  the  entire  system  will 
meet  the  reliability  specifications.  The  production  of  such  parts 
requires  much  more  than  improved  quality  control  on  the  production 
line  and  better  testing  procedures.  Reliability  must  be  designed  Into 
the  parts . 

In  order  to  design  parts  which  are  reliable  it  1b  necessary  to 
understand  the  physical  processes  which  control  the  manufacture  and 
operation  of  the  component.  It  is  impossible  to  design  a  reliable 
part  if  one  does  not  know  what  to  do  to  make  it  reliable.  Unfortunately, 
with  many  devices  such  as  transistors,  it  seems  that  not  enough  is 
known  about  modes  of  failure.  Only  after  a  considerable  understanding 
of  the  nature  of  failures  modes  and  wear-out  has  been  accumulated  is 
it  possible  to  design  with  confidence  a  part  and  production  process 
to  obtain  the  desired  reliability.  Many  things  are  known  today  vhich, 
if  used,  could  improve  the  reliability  of  such  devices  considerably. 

The  production  process  itself  is  very  important  in  obtaining  high 
reliability  parts.  A  production  process  which  giveB  vide  variability 
is  not  satisfactory  even  though  some  of  the  parts  may  be  very  reliable. 
The  reason  for  this  is  that  it  appears  to  be  very  difficult  to  determine 
whether  or  not  a  part  will  be  reliable  just  by  making  some  simple  tests 
on  it  Immediately  after  it  is  produced.  One  cannot  rely  60lely  on  a 
testing  program  to  choose  reliable  parts  from  a  population  with 
considerable  variance. 


Quality  control  is  needed  at  each  important  step  in  the  production 
process.  It  is  much  easier  to  catch  something  going  wrong  at  some 
intermediate  step  then  it  is  in  testing  finished  parte  because  a 
very  large  number  cf  finished  parts  may  need  to  be  tested  to  find 
defects  which  could  be  caught  by  making  relatively  few  tests  at  an 
intermediate  stage. 

In  order  to  obtain  components  with  the  desired  reliability  (no 
wear-out  or  drift  out  of  tolerance  for  given  time)  the  purchaser 
must  be  able  to  write  suitable  specifications  for  the  component 
manufacturer.  To  be  able  to  do  this  requires  the  ability  to  perform 
the  tasks  outlined  in  the  previous  sections.  In  addition,  there  must 
be  a  close  working  reiationsliip  between  the  component  manufacturer  and 
the  system  manufacturer .  The  most  important  contribution  of  this 
relationship  would  oe  better  definitions  of  tolerances.  For  many 
technical  and  other  reasons,  nearly  all  tolerances  ore  not  reasonable 
or  practical. 


Reliability  Testing 

Reliability  testing  is  an  important  facet  in  the  manufacture  of  reliable 
systems.  Many  problems  arise  in  reliability  testing,  hovever.  We  shall 
c'-'n.-.Mer  tvo  aspects  of  the  testing  problem  here:  (l)  testing  of 
component  parts,  (2)  testing  of  final  systems.  It  seems  to  be  a  quite 
popular  philosophy  today  to  believe  that  the  reliability  of  a  component 
part  of  system  will  be  verified  by  some  sort  of  testing  procedure. 


Consider  first  the  problem  of  testing  component  parts.  After  these 
ports  have  been  designed  and  manufactured,  hov  do  ve  know  that  they 
actually  meet  the  reliability  specifications?  Usually  the  procedure 
taken  is  to  place  enough  units  on  test  in  order. to  estimate  if  mean 
time  to  feui lure  of  the  component  distribution  is  vithin  a  given  interval 
vith  some  degree  of  confidence.  For  a  component  which  is  supposed  to 
operate  successfully  with  a  high  probability  for  a  long  period,  such 
as  five  years,  several  problems  are  encountered  in  developing  a 
testing  program.  First  there  is  generally  not  enough  time  available 
to  test  any  single  component  for  a  five  year  period.  Facing  this  fact, 
the  argument  is  often  developed  that  an  estimate  of  the  mean  time  to 
failure  with  the  required  degree  of  confidence  can  be  obtained  by 
testing  much  larger  quantities  of  parts  for  much  shorter  periods  of 
time.  This  argument  is  valid  if  the  component  actually  follows  the 
exponential  failure  distribution.  If  the  failures  are  completely 
random,  then  a  trade-off  between  the  number  of  parts  tested  and  time 
is  possible.  Suppose,  however,  that  a  wear  out  mechanism  sets  in  after 
three  years  and  every  part  will  fail  after  four  years.  This  would 
never  be  determined  by  testing  parts  for  only  six  months  or  a  year. 

It  i6  a  very  dangerous  procedure  to  try  to  compress  the  time  scale  on 
testing  without  sufficient  evidence  that  this  is  valid. 

let  us  assume  that  the  component  does  obey  the  exponential  failure 
distribution  and  that  ve  can  test  for  a  shorter  period  by  placing  more 
units  on  test.  A  simple  computation  show,  that  if  components  which 
are  supposed  to  have  a  very  long  mean  time  to  failure  are  to  be  tested 
for  times  not  long  compared  to  the  mean  time  to  failure,  and  it  is 
desired  to  obtain  an  accurate  estimate  of  the  mean  time  to  failure,  the 
number  ox'  units  which  must  be  tested  can  easily  run  into  tens  of 
thousands  or  even  hundreds  of  thousands  depending  on  how  accurately  one 
desires  the  estimate  to  be.  Clearly  the  amount  of  money  required  to 
carry  out  such  a  testing  program  on  a  number  of  components  is  huge. 
Furthermore,  if  the  components  cannot  be  made  on  a  high  production 
basis,  it  may  take  the  manufacturer  years  to  make  enough  parts  for  the 
test  program  alone. 

Finally,  let  us  ask  vhat  useful  information  is  gained  from  the  type  of 
test  program  outlined  above.  Provided  all  the  assumptions  made  are 
valid,  we  do  indeed  obtain  on  estimate  of  the  mean  time  to  failure. 
Hovever,  it  tells  us  nothing  about  the  variability  in  the  manufacturing 
process  with  time.  It  doeG  not  follow  that  the  manufacturing  process 


will  always  yield  6uch  good  parts.  The  process  may  get  out  of  control 
and  In  components  will  not  be  as  reliable  as  necessary.  Vhen  such 
hugh  +  .sting  programs  are  required,  it  is  practically  impossible  to 
carry  out  the  test  once,  let  alone  periodically,  to  make  sure  the 
production  process  is  under  control.  Hence,  it  would  appear  that  one 
must  face  the  fact  that  it  will  be  extremely  difficult  to  oe  sure  the 
component  vendor's  production  process  is  under  control  simply  by  using 
statistical  testing  procedures. 

Let  us  next  examine  the  system  test  problems .  How  do  we  know  that  the 
systems  meet  the  reliability  specifications?  In  many  instances  it  is 
impossible  to  devise  tests  which  will  simulate  actual  operating 
conditions  and  which  will  not  destroy  the  systems.  Even  when  it  is 
possible  to  realistically  simulate  operating  conditions,  the  number  of 
systems  which  must  be  tested  and  the  length  of  time  over  which  they  would 
need  to  be  tested  make  the  task  of  demonstrating  that  the  systems  have 
the  desired  reliability  practically  impossible.  This  is  especially  true 
when  only  a  very  small  number  of  systems  are  to  be  built.  For  example, 
if  two  guidance  and  control  systems  were  manufactured  for  use  on  two 
space  vehicles,  it  is  not  possible  to  prove  ahead  of  time  much  of 
anything  about  whether  or  not  these  two  systems  meet  the  reliability 
requirements.  The  final  proof  is  in  their  operational  behavior.  It 
seems  to  be  a  property  of  many  systems  that  we  cannot  really  eiy  much 
about  their  reliability  until  they  get  into  operational  use. 

The  above  discussion  suggests  that  the  normal  statistical  experiments 
become  hopelessly  costly  and  time  consuming  or  are  otherwise  impractical 
when  they  are  used  to  attempt  to  demonstrate  the  reliability  of 
components  or  systems.  It  appears  that  it  will  be  economically 
impossible  to  statistically  demonstrate  the  reliability  of  components 
and  systems  with  any  high  degree  of  confidence  or  regularity.  There¬ 
fore,  some  other  more  certain  and  practical  procedure  must  be  used. 

This  assurance  should  perhaps  come  from  careful  design  and  manufacture 
with  an  emphasis  on  extra  systems  for  functional  tests  of  physical  and 
electronic  phenomenon  considered  critical,  unique,  or  advanced. 

One  of.  the  most  promising  areas  for  testing  seems  to  be  in  the  area  of 
Identifying  defective  parts  or  systems  vith  a  relatively  short  test. 

The  objective  is  to  test  every  component  and  system  and  to  eliminate 
those  which  axe  defective  without  harming  the  others.  Such  tests  are 
often  referred  to  as  "burn-in"  tests.  To  develop  such  tests  it  is 
necessary  to  have  a  thorough  understanding  of  the  modes  of  failure  for 
the  component  or  system  and  the  types  of  defects  which  are  most  likely 
to  occur-  Today  not  too  much  is  known  about  suitable  burn-in  test6  either 
for  components  or  systems. 

Another  area  which  ie  receiving  considerable  attention  is  that  of 
accelerated  life  tests.  Attempts  are  being  made  to  develop  tests  which 
increase  the  severity  of  some  conditions  such  as  temperature,  and 
thereby  introduce  a  failure  in  a  much  shorter  time  than  under  normal 
conditions.  Tie  whole  idea  is  to  be  able  to  develop  accelerated  tests 
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which  will  correlate  vith  failures  under  normal  conditions.  The  job 
of  developing  accelerated  tests  is  a  very  touchy  one,  especially  if 
there  is  to  be  a  very  large  contraction  in  the  time  scale.  Hie 
difficulty  lies  in  the  fact  that  the  accelerated  and  normal  life 
tests  cannot' be  conducted  on  one  and  the  sarae_component .  For  the 
accelerated  life  tests  to  be  meaningful  at  all,  the  failure  mechanism 
under  accelerated  conditions  must  not  differ -from  those  under  normal 
conditions.  Even  if  this  condition  is  not  (and  the  task  of  demon¬ 
strating  it  is  met,  can  be  very  difficult),  the  tests  are  often  not 
practical.  This  is  because  extremely  large  samples  must  be  tested 
to  obtain  an  accurate  estimate  of  the  reliability  of  the  components 
under  normal  conditions  when  the  time  contraction  is  very  large. 


MANAGEMENT  ASPECTS  ~ 

The  company  confronted  with  meeting  a  stringent  reliability  requirement 
for  the  first  time  faces  many  management  problems.  There  are  financial 
problems  caused  by  increased  costs  in  rather  obscure  and  intangible 
areas.  There  are,  of  course,  engineering,  manufacturing  and  quality 
problems  as  outlined  above.  In  addition,  questions  arise  in  purchasing 
concerning  vendor- vendee  relationships;  in  industrial  relations  on 
the  always  important  questions  of  skill,  training,  seniority;  and  in 
administration  on  how  to  organ! ze  for  reliability.  In  short,  all 
areas  of  activity  are  affected. 


The  challenge  of  reliability,  to  date,  has  been  met  primarily  by 
engineering  and.  quality  control.  However,  some  companies,  recognizing 
the  more  general  nature  of  the  problem,  have  focused  attention  on 
organization  and  the  placing  of  responsibility  for  reliability.  When 
such  responsibility  has  been  delegated  it  may  be  found  in  a  variety  of 
places.  Some  of  these  appear  to  be  more  political  rather  than 
practical  expedients.  In  certain  instances  quality  control  is 
charged  with  the  entire  responsibility  for  reliability.  In  others, 
the  reliability  function  is  to  be  found  completely  within  engineering; 
or  a  committee  consisting  of  members  from  many  departments  attempts 
to  advise  someone  in  higher  management  on  what  actions  should  be  taken. 
None  of  these  seems  to  be  very  effective.  Quality  control  cannot 
effectively  supervise  reliability  activities  in  engineering.  Equally 
well,  engineering  usually  cannot  implement  a  reliability  program  in 
manufacturing.  There  is  a  further  difficulty  in  having  the  reliability 
responsibility  lie  in  a  single  organization.  In  such  cases  it  is 
quite  possible  that  the  reliability  function  will  be  used  to  serve 
the  objectives  of  that  particular  department  and  will  not  be  effective. 
For  example,  in  many  companies  salary  levels  are  based  upon  numbers 
of  persons  supervised.  A  zealous  manager  may  tenaciously  maintain 
control  over  the  He liability  Group  for  this  reason  although  it  might 
be  more  effective  in  s^'ie  other  department. 

The  management  aspects  of  reliability  will  be  discussed  in  two  parts; 
(l)  the  short  range  requirements,  and  (2)  the  long  range  requirements. 
These  tend  to  be  quite  different  because  of  the  existing  nature  of  the 
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We  shall  begin  with  a  discussion  of  the  type  of  reliability  organisation 
which  might  effectively  meet  the  challenge  of  today.  Such  an 

*  organization  must  be  strong  and  its  manager  must  have  the  power  to 
make  decisions  and  to  enforce  these  in  any  area  where  the  need 
arises.  In  many  respects  reliability  people  are  in  the  same  state 
today  that  quality  control  vas  some  years  ago  before  it  really  obtained 

#  authority  to  stop  production  when  quality  was  poor.  Until  the 
reliability  organization  has  the  authority  to  make  changes,  it  will 
be  unable  to  make  an  effective  contribution. 

Clearly  no  reliability  program  can  succeed  unless  the  top  management 
is  convinced  of  the  need  for  it.  When  the  management  becomes  so 
convinced,  as  ultimately  it  must  be  if  the  organization  it  directs  is 
to  survive,  the  way  is  open  to  develop  a  valuable  reliability  program. 
Its  first  task  will  be  to  delegate  to  some  group  or  groups  the 
primary  responsibility  for  protecting  the  company's  interests  in 
meeting  reliability  specifications.  The  responsibility  for  reliability 
cannot  be  suitably  delegated,  per  se,  to  any  one  line  organization, 
especially  since  it  must  deal  with  almost  all  departments.  Organi¬ 
zationally,  it  might  be  better  to  have  a  staff  group  responsible  for 
reliability  assigned  to  the  office  of  the  Chief  Executive  who  operates 
the  Company  or  Division.  Perhaps  in  some  cases,  even  at  a  higher 
level.  The  proper  Jurisdiction  of  this  group  "is  the  most  controversial 
subject  in  reli  jility  today.  It  even  transcends  the  technical, 
discussions  revolving  about  the  mathematical  concepts  of  testing 
and  assurance. 

Regardless  of  the  above,  for  present  day  management  structures,  it  is 
best  for  the  reliability  group  to  report  to  some  line  organization. 

If  management  were  sufficiently  aware  of  the  Importance  of  this  group, 
as  pointed  out  above,  it  would  no  doubt  report  to  the  Chief  Executive. 
However,  since  this  enlightenment  will  be  some  years  in  coming,  the 
reliability  group  should  report  to  the  first  level  of  management  in 
the  Company  or  Division.  Because  of  the  responsibilities  involved  it 
should  not  report  at  any  lover  level.  The  choice  of  line  functions 

\  lies  between  Engineering  and  Quality  Control.  Hxis,  of  course,  assumes 

an  independent  Quality  Control  Department  which  reports  to  the  Chief 
Executive  of  the  Company. 

The  state-of-the-ar L  or  sophistication  of  the  company's  products  and 
management  struct^sre  will  dictate  the  final  selection.  For  crash 
programs  of  significant  size  and  complexity,  engineering  is  the  most 
likely  choice.  Hus  is  mainly  because  quality  control  needs  technical 
upgrading  and  education  as  well  as  all  other  departments.  However,  for 
programs  incorporated  into  company  activities  under  normal  conditions, 
quality  control  should  be  the  department  selected.  The  reasons  are 
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that  objectivity  and  efficiency  are  of  prime  importance.  Quality 
Control  departments  are  by  nature  more  orientated  in  these  directions 
than  are  Engineering  departments. 

management  of  the  reliability  function  should  not  be  delegated  to 
a  coranittee.  The  connittee  method  is  satisfactory  for  coordination 
and  liaison  needs.  However;  it  cannot  be  relied  upon  to  Bupply  the 
drive  necessary  for  a  successful  reliability  program.  What  is  needed* 
Is  an  aggressive  forceful  approach  aimed  at  uniting  and  educating  all 
functions  regarding  their  duties  and  responsibilities  concerning  this 
nev  challenge . 

Regardless  of  the  function  to  vhich  it  reports.  Reliability  vill  be 
categorized  as  a  staff  organization.  However,  the  head  of  the 
reliability  organization  must  in  reality  be  able  to  make  decisions 
and  have  them  enforced.  Such  an  arrangement  is  not  too  uncommon. 

Many  so-called  staff  organizations  do  in  reality  make  decisions. 

Hie  reliability  organization  has  a  number  of  responsibilities  extending 
from  development  of  components  to  the  study  of  the  operational  behavior 
of  the  final  system.  Briefly,  some  of  their  responsibilities  are: 

1 .  Components : 

Set  up  reliability  specifications  on  components  to  be 
supplied  by  vendors.  It  was  noted  above  that  in  many 
cases  It  is  very  difficult  to  test  components  to 
determine  their  reliability.  Hius,  the  reliability 
organization  must  be  able  to  have  some  control  over  the 
vay  in  vhich  the  component  supplier  designs  and 
manufactures  these  components.  Hie  reliability  group 
must  also  be  involved  with  decisions  in  packaging, 
transporting  and  storing  the  components.  It  should 
have  some  control  over  the  purchasing  department  so 
that  a  number  of  standard  practices  such  as  buying  from 
a  component  vendor,  simply  because  it  is  the  lowest  bid, 
will  be  abolished. 

2.  Circuit  Design: 

The  reliability  group  must  monitor  the  design  of  the 
system's  circuit.  In  many  places  design  review  is 
becoming  popular.  Tills  appears  to  be  necessary  at  the 
present  time.  It  will  also  probably  be  necessary  for 
the  reliability  organization  to  establish  a  standard 
parts  list  and  to  prevent  use  of  any  parts  not  on  the 
lust  unless  approved  by  the  reliability  group.  This  is 
necessary  in  order  to  concentrate  on  improving  a 
relatively  small  number  of  components.  Sometimes  there 
is  a  certain  sacrifice  made  by  restricting  the  number 
of  parts,  but  this  is  more  11.  m  made  up  for  by  the 
concentrated  effort  made  to  improve  the  reliability  of 
these  parts. 
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3*  Manufacturing: 

The  reliability  group  must  joake  sure  that  the  manufacturing 
process  yeilds  a  system  vith  the  highest  possible  reliability. 
It  must  also  be  sure  that  practices  which  are  detrimental  to 
reliability  are  not  allowed.  For  example,  if  manufacturing 
runs  out  of  inventory  on  some  component,  it  must  not  be 
allowed  to  go  out  on  its  own  and  purchase  locally  some 
similar  component  which  has  a  much  lower  reliability.  This 
is  much  more  than  a  quality  control  function.  Naturally,  - 
the  quality  control  organisation  will  be  expected  to  handle 
the  details  of  testing,  etc.,  to  insure  that  a  product  of 
suitable  quality  is  being  produced. 

4.  Failure  Reporting  and  Analysis: 

The  task  of  setting  up,  coordinating  and  monitoring  a  failure 
reporting  and  failure  analysis  system  rests  in  large  part 
with  the  reliability  organisation.  It  is  desirable  to  have 
a  procedure  such  that  each  time  a  failure  occurs,  it  is 
properly  recorded  and  a  detailed  study  of  the  component  or 
circuit  is  made  to  determine  why  the  failure  occurred  and 
what  can  be  done  to  correct  it.  An  integrated  failure 
reporting  system  which  extends  all  the  way  from  the  component 
manufacturers  to  operational  systems  is  a  most  important 
facet  of  a  reliability  program. 

5.  Education: 

As  has  been  noted  previously,  an  understanding  of  reliability 
problems  is  woefully  lacking  at  almost  all  levels  in  many 
companies.  Hence  one  of  the  most  important  tasks  of  the 
reliability  group  is  that  of  education.  Management  must 
be  educated,  designers  must  be  educated,  and  60  must 
manufacturing  and  purchasing.  Even  suppliers  and  customers 
need  education  in  certain  cases.  Naturally  the  type  of 
education  required  .differs  somewhat  from  group  to  group, 
viz:  from  management  to  designers.  Once  some  of  the 
educational  goals  have  been  accomplished,  the  burden  falling 
on  the  reliability  group  should  be  considerably  lightened. 

6.  Analysis: 

It  was  indicated  in  the  technical  discussion  that  a  great 
deal  of  work  needs  to  be  done  in  the  area  of  quantitative 
reliability  analysis.  In  general,  designers  or  engineers 
normally  found  in  an  electronics  company  do  not  have  the 
proper  background  in  statistics,  numerical,  analysis,  etc., 
to  allow  them  to  work  in  this  area.  Consequently  the  task 
of  developing  these  analytical  techniques  falls  to  a  large 
extent  upon  the  reliability  group. 


Needless  to  say,  it  cannot  be  expected  that  all  the  above  tasks  can  be 
carried  out  by  the  reliability  group  alone.  They  must  be  done  in 
cooperation  with  other  departments.  It  should  be  noted,  hovever,  that 
the  above  tasks  require  differing  types  of  people.  No  single 
individual  could  be  expected  to  work  efficiently  upon  all  of  them. 

Some  jobs  require  a  very  practical  approach,  while  others  require  a 
highly  mathematical  approach.  Thus,  one  would  expect  to  find  & 
variety  of  skills  within  the  reliability  group.  In  the  long  run  the 
areas  of  responsibility  will  not  change  too  much.  However,  it  may 
not  be  necessary  to  have  a  single  reliability  organization  managing 
all  of  them. 

The  Long  Range  Organization 

Reliability  is  Just  another  quality  characteristic.  In  fact,  it  only 
lends  the  dimension  of  time  to  any  other  dimension  or  functional 
characteristic.  For  this  reason,  reliability  will  eventually  take 
its  place  along  with  all  others  in  the  part  or  product  specifications. 
It  will  be  dealt  with  in  much  the  same  manner  as  industry  today  deals 
with  any  other  quality  characteristic.  Engineering  will  be  responsible 
for  its  incorporation  into  the  product  and  processing  specifications. 
Manufacturing  will  be  responsible  for  the  reliability  of  parts, 
components,  and  systems  produced.  Quality  Control  will  be  responsible 
for  the  measurement  of  reliability  to  assure  that  the  specification 
was  met. 

For  the  long  range  program,  the  main  change  will  be  in  management's 
attitude  tovard  some  of  the  so-called  non-productive  functions  of  the 
organization.  The  three  areas  slated  for  the  most  attention  are 
quality  control,  purchasing,  and  certain  engineering  services  such 
as  components  and  materials,  and  standards  groups.  The  emphasis  will 
be  on  education  and  strict  adherence  to  Jurisdictional  lines  for  the 
discharge  of  responsibilities.  Progressive  organizations  will  have 
extensively  supplemented  formal  education  programs  with  in-plant 
training  programs.  For  example,  in  some  years  to  come,  all  employees 
in  a  large  electronic  company,  from  the  lowest  level  of  supervision 
up,  will  have  the  equivalent  of  a  BSEE  today.  In  addition,  they  will 
have  had  extensive  formal  training  in  the  technical  aspects  of  quality 
control,  procurement,  and  industrial  relations.  Today  this  need  for 
cross-po Ionization  is  very  acute.  It  is  the  principle  reason  for 
"projectizing"  such  functions  as  reliability. 

With  on  improvement  in  the  general  level  of  technical  understanding 
for  the  various  functions,  lines  of  responsibility  may  be  more  clearly 
defined.  Authority  will  oe  delegated  to  departments  which  can 
discharge  the  responsibility  most  efficiently.  The  role  of  technical 
personnel  will  ue  one  more  of  support  rather  than  action. 

For  instance,  it  will  be  possible  to  delegate  to  quality  control  the 
responsibility  for  measuring  reliability.  Such  delegation  will 
penetrate  all.  areas  of  the  company  including  research  and  development. 
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Since  the  quality  control  specialists  vill  then  be  able  to  understand 
the  scientific  explanation  of  the  phenomenan  to  be  measured,  the  tack 
of  making  such  measurements  may  be  delegated  to  quality  control. 

t-i  "xaaples,  such  as  the  above,  vhere  the  very  existence  of  the  company 
or  an  important  project  depends  upon  the  accuracy  of  facts  and  their 
analysis,  it  may  be  necessary  to  so  divide  the  tasks.  Engineering  or 
research  responsible  for  creation,  quality  responsible  for  measuring, 
testing,  and  analyzing. 

Similarly,  with  procurement,  reliability  will  present  many  interesting 
changes  to  modes  and  methods  of  operation.  The  improvement  of 
components  which  are  used  in  advanced  electronic  systems  will  affect 
not  only  engineering,  procurement,  manufacturing,  and  quality  control, 
b\>t  also  fiscal  and  legal  departments.  The  task  of  improving  the 
components  which  go  into  electronic  systems  will  fall  upon  the 
component  manufacturers.  However,  the  funds  required  for  improvement 
programs  may  in  many  cases  come  to  the  component  manufacturer  from 
the  system  manufacturer  in  the  form  of  contracts.  This  means  that  the 
system  manufacturer  will  want  to  have  some  control  over  what  the 
component  supplier  does.  In  turn,  this  implies  that  the  system 
manufacturer  will  want  to  know  something  about  the  manufacturing 
process  for  the  components.  The  system  manufacturer  may  even  wish 
to  go  so  far  as  to  have  its  engineers  examine  the  component  production 
process  periodically  to  make  sure  that  high  reliability  standards  are 
being  maintained.  Such  an  arrangement  is  completely  contrary  to  the 
present  state  of  affairs  where  component  manufacturers  tend  to  be 
rather  secretive  about  their  manufacturing  process  or  at  least  parts 
or  it. 

Furthermore,  in  many  cases  the  money  for  improving  components  may 
come  from  the  military  --  either  in  direct  contracts  to  the  component 
manufacturer  or  indirectly  from  the  system  manufacturer.  Quite 
possibly  the  government  vill  require  that  a  lot  more  information  be 
released  for  general  use  than  the  component  vendor  would  like. 

Because  of  the  above  requirements,  one  can  expect  that  a  new 
relationship  vill  be  required  between  the  component  vendors  and  users. 
One  can  go  cn  and  ask  what  vill  happen  to  those  component  manufacturers 
which  do  not  receive  large  contracts  to  improve  their  parts.  They 
will  either  be  forced  to  use  their  own  money  for  improvement  or  be 
faced  with  loss  of  markets.  Here  again  there  ore  many  problems  to  be 
faced. 


SUMMARY 


The  gist  of  the  asove  is  that  organization  for  reliability  in  the 
short  range  is  being  done  along  project  lines.  This  is  caused  by  the 
present  difference.:  .n  levels  of  technical  ability  of  tne  various 
departments  or  gr  up.  involved.  Over  the  long  range,  the  need  for  a 
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special  group  called  "Reliability"  vill  vanish  as  standard  departmental 
functions  become  able  to  absorb  their  responsibilities.  This  will 
became  possible  as  the  technical  levels  of  all  functions  improve 
until  finally  all  engineers  and  specialists  in  purchasing,  quality 
control,  and  manufacturing  are  more  knowledgeable  in  electronics 
methodology.  At  that  future  time,  organization  charts  will  look  much 
as  they  do  today,  and  reliability  will  be  another  characteristic 
specified  by  engineering  and  measured  by  quality  control. 


R  AND  D  ENGINEERING  RELIABILITY  PROGRAM 


AT  MOTOROLA'S  WESTERN  MILITARY  ELECTRONICS  CENTER 


Fred  E.  Dreste  and  Charles  A.  Krohn 


Motorola  Inc. 

Western  Military  Electronics  Center 
Scottsdale  (Greater  Phoenix),  Arizona 


INTRODUCTION 


In  the  past  decade  reliability  has  been  properly  recognized  as  one  of 
the  major  factors  that  determines  the  worth  of  electronic  equipment.  We 
are  glad  to  see  that  reliability  is  now  considered  simultaneously  with 
other  worth  factors  such  as  performance,  weight  and  size,  maintaina¬ 
bility,  cost,  operability,  and  producibllity. 

Motorola  recognizes  the  importance  of  reliability  in  military  elec¬ 
tronic  equipment.  We  have  been  Increasing  reliability  through  both  in¬ 
ternal  programs  and  participation  in  external  industry-wide  programs. 
Our  approach  for  achieving  an  acceptable  reliability  level  within  econom¬ 
ic  limits  is  detailed  in  the  following  discussion. 

TOP  MANAGEMENT  SUPPORT 


Top  management  at  Motorola  has  long  recognized  the  necessity  for 
increasing  the  reliability  of  military  electronic  equipment.  Of  more 
significance  has  been  the  willingness  to  finance  reliability  effort  prior 
to  such  effort  being  contractually  required. 

Dr.  Daniel  Noble,  Executive  Vice  President  of  Motorola,  is  well-known 
for  his  speeches  and  publications  on  Reliability.  Mr.  Joseph  Chambers, 
Vice  President  and  General  Manager  of  WMEC,  was  the  original  Chair¬ 
man  of  the  Department  of  Defense  Adhoc  Group  on  Reliability  of  Elec¬ 
tronic  Equipment  and  recently  was  industry’s  representative  on  the 
Congressional  Advisory  Committee  on  Reliability  to  the  House  Committee 
on  Appropriations.  Several  papers  on  reliability!  have  been  presented 
and  published  by  Dr.  Robert  Samuelson,  Assistant  General  Manager. 
Mr.  A,  S.  Hume,  Manager  of  our  Communications  and  Navigation  Labora¬ 
tory,  was  Task  Chairman  of  Task  Group  2  for  the  Advisory  Group  on 
Reliability  of  Electronic  Equipment  Report  (AGREE),  June  4,  1957. 


1-52 


Other  Motorola  personnel  are  members  of  and  in  some  cases  the 
chairmen  of  various  reliability  and  components  committees  in  the  Aero¬ 
space  Industries  Association,  Electronics  Industries  Association,  In¬ 
stitute  of  Radio  Engineers,  and  American  Society  for  Quality  Control. 
Also,  many  papers  on  reliability  have  been  presented  and  published  by 
our  Chief  Engineer,  Reliability  and  Components  Group  Head,  and  several 
other  Reliability  and  Components  Group  personnel  (See  References). 

ORGANIZATION 

Our  particular  approach  to  reliability  is  designed  to  blend  with  our 
basic  organization  and  with  the  type  of  business  we  seek.  Motorola's 
Western  Military  Electronics  Center  (WMEC)  at  Scottsdale  (Greater 
Phoenix),  Arizona  is  one  of  3  primary  Motorola  Military  activities. 
Other  activities  are  the  Chicago  Military  Electronics  Center  (CMEC) 
and  at  Riverside,  California,  the  Systems  Research  Laboratory.  The 
Phoenix  operation  is  primarily  concerned  with  electronic  equipment  for 
manned  and  unmanned  airborne  vehicles,  and  the  associated  support 
equipment.  Our  customers  include  all  services,  either  directly  or  in¬ 
directly  through  other  contractors,  and  Governmental  Agencies. 

At  WMEC  we  have  the  departmental  breakdown  by  basic  functions  into 
Contracts,  R  and  D  Engineering,  Purchasing,  Manufacturing,  Quality 
Control,  and  other  typical  groupings.  Figure  1  illustrates  this  organiza¬ 
tion.  The  R  and  D  Engineering  Department  is  fundamentally  on  a  project 
basis.  Current  total  engineering  manpower  is  around  900.  Management 
firmly  believes  that  a  strong  project  approach  with  a  design  team  devoted 
to  a  single  project  is  optimum.  This  approach  is  shown  in  Figure  2. 
There  are  3  R  and  D  Laboratories,  where  each  Laboratory  has  4  or  5 
Sections  and  each  Section  4  or  5  Projects.  Supporting  the  R  and  D  Pro¬ 
jects  are  centralized  Staff  and  Service  Groups,  which  have  specialists 
and  facilities  in  areas  necessary  to  sustain  the  R  and  D  operating  pro¬ 
jects. 

Reliability  functions  in  the  R  and  D  Engineering  area  are  centralized 
in  the  Reliability  and  Components  Group.  Figure  3  illustrates  this  group's 
functional  organization.  The  reliability  functions  of  the  Reliability  and 
Components  Group  (ns  shown  in  Figure  3)  will  be  discussed  subsequently 
and  as  pertinent. 
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RELIABILITY  EXPERIENCE 

Motorola's  ability  in  the  area  of  reliability  is  a  result  of  direct 
experience  as  well  as  management  support  of  general  reliability  pro¬ 
grams.  Considerable  experience  in  attaining  reliability  has  been  gained 
from  our  standard  reliability  program.  This  standard  reliability  pro¬ 
gram  is  a  basic  effort  that  reaches  all  projects,  and  is  presented  in  the 
fallowing  section.  Where  the  customer  recognizes  the  reliability  need 
through  additional  funding  specifically  to  achieve  reliability,  further 
effort  above  the  basic  program  is  applied.  Several  such  programs  where 
additional  reliability  effort  was  funded  are  cited  below. 

Several  years  ago  we  re-engineered  an  existing  electronic  control 
system  used  in  the  Navy's  Terrier.Surface-to-Air  missile.  2  The  redesign 
was  aimed  expressly  at  higher  reliability,  and  results  were  significant. 
During  design  approval  flights  at  the  Naval  Ordnance  Test  Station,  China 
Lake,  California,  no  failure  was  ever  attributed  to  the  Motorola  designed 
control.  Concentrated  reliability  effort  is  presently  being  applied  on 
such  important  efforts  as  the  Guidance  Beacon  for  the  Bomarc  missile, 
B-58  IFF  equipment,  the  guidance  unit  for  the  Sidewinder  missile,  and 
Command  Receivers  for  the  Man-in-Space  project  and  the  Minuteman. 

STANDARD  RELIABILITY  PROGRAM 


A  feasible  plan  for  achieving  acceptable  reliability  within  economic 
limits  is  incorporated  in  our  standard  reliability  program  that  is  out¬ 
lined  below.  The  standard  reliability  program  is  a  minimum  effort  that 
directly  and  indirectly  reaches  all  projects. 

Standard  Reliability  Program 


A.  Reaches  All  Projects 

B.  No  Additional  Project  Funding 

C.  Basic  Reliability  Approach 

1.  Use  Reliable  Parts 

2.  Design  for  Reliability 

3.  Test  to  Uncover  Weaknesses 

D.  Reliability  Indoctrination  Efforts 

E.  Reliability  Control  Measures. 


Several  elementary  but  fundamental  points  represent  the  starting 
place  of  the  development  of  our  reliability  program.  The  worth  of 


electronic  equipment  is  determined  by  several  factors — performance, 
weight,  size,  cost,  operability,  maintainability,  reliability, produclbility, 
and  safety.  In  each  individual  equipment  the  various  factors  will  often 
be  different  in  importance.  For  the  reliability  factor,  we  have  a  basic 
efiort  that  reaches  all  projects.  In  addition  to  this  basic  effort,  reliability 
efforts  are  emphasized  where  the  individual  situation  warrants.  Thus 
reliability  takes  its  proper  perspective  in  the  total  viewpoint. 

In  the  development  era  of  electronic  equipment  several  elements  are 
highly  significant  in  achieving  reliability.  Our  development  reliability 
program  entails  the  pursuit  of  these  elements,  which  are: 

1.  Use  reliable  component  parts, 

2.  Design  electrically  and  mechanically  for  reliability, 

3.  Test  early  to  uncover  weaknesses. 

In  our  basic  reliability  program  the  project  design  engineers  are  made 
aware  of  the  importance  of  these  elements,  and  are  assisted  in  properly 
applying  these  elements.  Each  of  these  elements  is  discussed  below. 

The  responsibility  for  achieving  reliability,  as  well  as  for  achieving 
performance,  remains  with  the  individual  who  can  best  achieve  desired 
resulta--the  project  design  engineer.  He  is  supported  by  company-wide 
programs  for  increasing  reliability  in  each  of  the  above  elements.  En¬ 
gineering  management  exerts  positive  control  on  project  reliability 
efforts  by  formally  reviewing  the  reliability  elements  at  periodic  project 
reviews.  Reliability  educational  efforts  are  significant  in  our  approach, 
and  are  discussed  in  a  subsequent  section. 

Use  Reliable  Component  Parts 

Selection  of  reliable  component  parts  and  provisions  for  ensuring  their 
high  quality  during  subsequent  procurement  are  prime  requisites  in 
developing  and  building  reliable  electronic  equipment.  The  component 
parts  aspect  of  our  standard  reliability  program  Is  oulined  below.  We 
believe  that  our  systems  and. circuits  engineers  should  not  have  to  be 
proficient  in  component  parts,  and  have  specialists  in  this  area.  Com¬ 
ponent  parts  activity  is  centralized  in  the  Reliability  and  Components 
Group  that  was  formed  in  1953  within  our  Terrier  Project  and  expanded 
into  a  service  for  all  projects  in  1955.  All  component  parts  have  been 
divided  among  8  Sections  where  each  is  responsible  for  testing,  applica- 
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tion,  procurement  documents,  and  standardization.  Eighteen  engineers 
and  20  technicians  comprise  the  component  parts  specialists  staff 
(see  Figure  3). 

t'se  Reliable  Parts 


A.  Centralized  Component  Part  Specialists 

1.  Application 

2.  Testing 

3.  Procurement  Documents 

B.  Determine  Best  Part  Types  and  Vendors 

1.  Past  Experience 

2.  Screening  Tests 

3.  Tests  to  Failure 

4.  Vendor  Facility  Surveys 

C.  Control  Vendors  with  Adequate  Procurement 
Specifications  and  Perform 

1.  Qualification  Tests 

2.  Lot  Acceptance  Tests 

3.  Requalification  Tests 

D.  Standardization  Program 

1.  MIL  for  standardization 

2.  Company  (standard  specs)  where  reliability  stressed 

E.  Project  Level  Implementation 

1.  Approach  Planned  at  Inception 

2.  Controlled  by  Management  and  Specialists  Thru  Reviews. 

Motorola  has  supported  an  extensive  centralized  parts  testingprogram, 
including  tests  to  failure,  multi-level  stress  testing,  and  each  of  which 
is  methodically  documented.  In  addition,  we  have  participated  in  various 
inter-company  sponsored  tests.  Visits  to  the  manufacturing  facilities  of 
the  various  suppliers  of  component  parts  have  provided  important  supple¬ 
mental  information.  Feedback  of  part  failure  information  from  Incoming 
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Inspection  and  the  manufacturing  process  is  handled  by  Quality  Control, 
and  represents  another  input  source  to  our  parts  specialists.  We  also 
report  results  of  the  above  part  information  sources  to  our  vendors  in 
an  attempt  to  secure  product  improvement.  Such  part  information  has 
led  to  the  objective  selection  of  part  types  and  vendors. 

Component  part  types  commonly  used  and  vendors  selected  by  the 
part  specialists  are  presented  to  our  projects  through  the  Motorola 
Military  Division  Standard  Parts  Catalog.  Two  classes  of  component 
parts  are  included,  strict  MIL  component  parts  and  parts  with  reliability 
significantly  Improved  over  the  MIL  level.  Strict  MIL  parts  are  pro¬ 
cured  to  the  MIL  documents.  We  know  that  many  parts  are  better  than 
the  MIL  reliability  level  from  the  results  oi  our  testing  program.  Mo¬ 
torola  has  prepared  its  own  specifications  for  the  better  than  MIL  parts.3 
These  specifications  follow  the  MIL  format,  but  have  more  rugged  tests 
and  test  sequences,  require  more  specimens,  allow  fewer  failures, 
tighten  electrical  requirements,  and  have  provisions  for  periodic  re¬ 
assurance  tests,  including  penalty  clauses  forfailure.  We  started  pre¬ 
paring  such  specifications  in  1957,  and  now  have  36  specifications  and  96 
associated  drawings.  As  state  of  the  art  MIL  specifications  are  issued, 
we  obsolete  our  company  specifications.  For  the  more  special  parts  not 
suitable  to  standardization,  as  a  minimum  we  prepare  a  drawing  with 
general  requirement  notes  and  perform  limited  evaluative  tests.  Where 
reliability  and/or  high  production  are  pertinent,  we  prepare  a  full  speci¬ 
fication  and  perform  the  qualification  tests. 

Each  project's  component  parts  approach  is  preliminarily  planned  at 
the  proposal  era,  and  is  firmly  planned  at  the  project's  inception.  Pro¬ 
ject  personnel  are  assisted  by  the  Reliability  and  Components  Group  in 
planning  a  parts  approach.  The  Standards  program  having  both  strict 
MIL  and  company  high  reliability  parts  allows  each  project  to  emphasize 
standardization  through  MIL  parts  or  to  emphasize  reliability  through 
high  reliability  parts.  Several  control  measures  exist  with  respect  to 
the  actual  implementation  of  a  project’s  planned  component  parts  ap¬ 
proach.  R  and  D  Engineering  Management,  at  the  Laboratory  Manager 
level,  approves  each  project’s  approach.  Each  project  periodically 
circulates  its  part  type  and  vendor  selection  to  the  Reliability  and 
Components  Group,  Quality  Control,  and  Purchasing  for  information 
and  comments.  Also,  copies  of  all  parts  purchase  orders  are  sent  to 
the  Reliability  an«l  Components  Group  for  information  and  comments. 


Design  Electrically  and  Mechanically  for  Reliability 

Reliable  equipments  result  from  thorough  electrical  and  mechanical 
engineering  that  provides  adequate  safety  margins  for  the  end-use  opera¬ 
ting  conditions.  In  our  previously  discussed  component  parts  efforts,  we 
have  learned  the  capabilities  of  the  parts  selected  for  use.  Reliable 
circuits  are  the  direct  result  of  adequate  allowances  for  the  limitations 
cl  the  basic  building  blocks,  component  parts.  Following  are  the  outline 
and  discussion  of  designing  for  reliability. 

Design  For  Reliability 

A.  Electrical  Design 

1.  Reliability  Estimates 

2.  Simplicity 

3.  Proven  Good  Circuits 

4.  Derating  of  Parts 

5.  Part  Variations 

6.  Redundancy  Where  Necessary. 

B.  Mechanical  Design 

1.  Minimize  Environmental  Effects 

2.  Balance  Among  Reliability,  Maintainability,  Cost, 
Producibility,  Operability,  Size  and  Weight. 

C.  Project  Level  Implementation 

1.  Design  Aids  Available 

2.  High  ME: EE  Ratio 

3.  Controlled  by  Management  and  Specialists  Thru  Reviews, 

Reliability  estimates  at  the  paper  design  stage  allow  comparison  of 
reliabilities  of  Different  approaches  and  indicate  the  degree  of  effort 
required  for  various  sections  of  the  equipment.  In  the  electrical  design 
area  reliability  is  enhanced  by  designing  circuits  that  are  tolerant  to 
appreciable  and  realistic  variations  in  part  values  and  supply  voltages, 
using  a  minimum  number  of  p.irts  adequately  electrically  and  en¬ 
vironmental!  v  derated,  and  coordinating  circuit  functions  to  achieve 
simplicity.  4-,)  Whenever  feasible,  designers  make  use  of  available  de¬ 
signs  which  have  proven  good  during  field  use.  Circuit  design  is  the 
primary  responsibility  of  the  project  engineers.  These  designers  are 
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educated  to  the  above  electrical  design  considerations  through  various 
mediums  subsequently  discussed  in  the  section  on  Reliability  Indoctrina¬ 
tion  Efforts.  The  project  engineer  is  assisted  by  being  provided,  when  he 
his  paper  design,  with  information  on  part  limitations,  parameter 
variations,  recommended  deratings,  and  other  application  data.  As  part 
of  our  standard  reliability  program,  such  guidance  information  is  con¬ 
tained  in  our  Reliability  and  Components  Handbook,  and  Standards  Cata¬ 
log.  -  - 


Particular  attention  is  given  to  mechanical  design  in  order  to  minimize 
the  effects  on  component  parts  caused  by  the  environments  of  tempera¬ 
ture,  humidity,  vibration,  shock,  pressure,  radiation,  and  dust.  We  have 
a  ratio  of  1  to  6  for  mechanical  to  electrical  engineers,  insuring  that 
environmental  stresses  are  adequately  considered.  Mechanical  engineers 
are  assigned  to  projects,  and  packaging  approaches  are  coordinated  by 
our  Mechanical  Engineering  Lab  Specialists.  The  Mechanical  Engineering 
Lab  consists  of  Specialists  (12)  in  thermodynamics,  structures,  shock, 
vibration,  acoustics,  and  theoretical  mechanics.  Mechanical  design 
guides  are  published  in  the  Mechanical  Engineering  Memoranda  Manuals 
as  guidance  for  project  mechanical  engineers.  ~  *™~  ~~~ 

Control  measures  on  actual  electrical  and  mechanical  designs  are  the 
periodic  project  reviews.  Periodically  throughout  a  project,  normally  at 
6  week  intervals,  the  Laboratory  Manager,  Chief  Engineer,  Section  Head, 
and  staff  groups'  representatives  review  each  project.  Pottrtial  design 
deficiencies  are  brought  to  light  for  early  correction. 

Test  Early  to  Uncover  Weaknesses 

Careful  attention  to  the  design  practices  just  described  greatly  re¬ 
duces  the  number  of  reliability  problems  showing  up  during  subsequent 
manufacturing  and  use.  Exhaustive  testing  of  breadboards,  and  early  and 
final  prototypes  uncovers  remaining  weak  areas  for  corrective  action 
before  production  release.  This  aspect  of  our  approach  to  reliability  is 
outlined  and  discussed  further  below. 


Test  For  Reliability 

—  ~  - - ■ 

p  A.  Locate  Weukm  ssos  for  Correction 

v 

1.  Bench  Tests 

2.  Environmental  Tests 


^ c .--e*  -jYv-fvvvisr  riy.f.ss.'.v  r^sssx^sa. 


B.  MTBF  or  Reliability  Test  (where  reliability  emphasized) 

C.  Project  Level  Implementation 

1.  Approach  Planned  at  Inception 

2.  Controlled  by  Management  and  Specialists  at  Reviews. 

Systems  breadboards,  using  the  same  component  parts  as  planned  for 
the  final  equipment,  are  assembled  and  bench  tested  early  in  the  program 
to  uncover  interaction  probems.  Critical  component  part  parameters  and 
supply  voltages  are  varied  in  value  during  equipment  bench  tests  to  help 
prove  adequate  performance  safety  margins.  Equipment  prototypes  are 
operated  at  environmental  extremes  to  demonstrate  endurance  against 
.'ailure.  Equipment  performance  variations  measured  during  both  bench 
and  environmental  tests  are  analyzed  with  the  aid  of  statistical  techniques 
to  establish  manufacturing  and  end  use  tolerances. 

As  with  reliable  design  practices,  testing  of  breadboards  and  proto¬ 
types  by  projects  is  controlled  by  periodic  project  reviews  attended  by 
engineering  management  and  staff  group  representatives. 

RELIABILITY  INDOCTRINATION  EFFORTS 

We  believe  that  reliability  indoctrination  is  an  important  part  of  a 
reliability  program.  Our  indoctrination  methods  include  both  direct  and 
indirect  approaches,  as  shown  in  the  following  outline. 

Emphasis  on  Reliability  Indoctrination 

A.  Reliability  and  Components  Handbook 

B.  Mechanical  Engineering  Manual 

C.  Inplant  Reliability  Course 

D.  Project  Reliability  Coordinators 

E.  R  and  C  Group  Reports 

F.  Junior  Engineering  Training  Program 

G.  College  Courses 

H.  Participation  in  Industry-Wide  Reliability  Efforts. 

Our  Reliability  and  Components  Handbook,  initially  issued  ir  1955  and 
periodically  revised,  is  personally  issued  to  every  engineer.®  To  date 
over  700  handbooks  have  been  issued,  with  around  300  of  these  going 
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outside  of  Motorola.  The  Table  of  Contents  of  our  handbook  is  shown  as 
Appendix  B.  An  extensive  transistor  circuit  design  check  list  recently 
issued  has  been  well  received  by  our  design  engineers.  Just  recently 
the  fourth  presentation  of  our  inplant  Reliability  Short  Course  was  com- 
pieied.  Course  content  is  shown  below. 

Content  of  Motorola  WMEC's  Reliability  Short  Course 

Session  la.  Introduction  to  Reliability 

Session  lb.  Component  Parts  Program 

Session  2a,  Parts  Procurement  Documents 

Session  2b.  Parts  Standards  Program 

Session  3  Mechanical  Aspects 

Session  4  Reliability  Analysis  I,  Reliability  Models  and  Measure¬ 
ment 

Session  5  Reliability  Analysis  II,  Component  Part  Value  Variations 
and  Appropriate  Circuit  Design  Techniques 

Session  6  Reliability  Analysis  III,  Part  Derating,  Redundancy,  and 
Reliability  Estimating. 

About  one-half  of  our  design  engineers  have  attended  the  course,  and  we 
plan  to  periodically  repeat  the  course. 

Reliability  Coordinators  (reliability  specialists  assigned  to  projects 
emphasizing  reliability,  discussed  below)  and  other  engineers  working  in 
reliability  areas  meet  every  2  weeks  for  several  hours.  At  these  meet¬ 
ings,  which  are  analogous  to  seminars,  pre-selected  reliability  topics 
are  presented  and  discussed.  These  meetings  help  keep  our  reliability 
engineers  abreast  o!  the  state  of  the  art  and  allow  them  to  swap  notes  on 
their  problems  and  solutions.  New  junior  engineers  are  assigned  to  the 
Reliability  and  Components  Group  for  3  to  4  weeks,  as  well  as  to  other 
R  and  D  Engineering  staff  and  service  groups.  Quality  Control,  and  Pro¬ 
duct  Engineering.  Engineers  completing  this  sequence  of  assignments 
obtain  an  excellent  appreciation  of  the  methods  and  value  of  engineering 
for  reliability  and  pruducibility.  Junior  engineers  who  will  be  designing 
circuits  attend  an  i-xt  >  nsive  10  week  semiconductor  circuit  design  course. 


Through  several  years  of  correspondence  and  discussions  we  have 
assisted  several  engineering  colleges  in  introducing  reliability  concept, 
into  their  curriculum.  ?  In  cooperation  with  the  University  of  Arizona, 
our  personnel  have  instructed  a  graduate-credit  course,  Reliability  and 
Ei.g..iCering  Statistics.  We  attend  and  often  present  papers  at  all  im¬ 
portant  reliability  symposia  and  conventions,  as  well  as  actively  partici¬ 
pate  in  several  AIA,  IRE/ASQC,  and  E1A  working  committees.  This  keeps 
us  well  informed  on  the  state  of  the  art  in  systems  and  component  part 
reliability. 

PROJECTS  EMPHASIZING  REL  [ABILITY 


Our  standard  reliability  program  enables  us~to  produce  more  reliable 
equipment  at  competitive  prices  even  though  reliability  has  not  been  em¬ 
phasized  by  the  customer.  Where  the  customer  recognizes  the  reliability 
need  through  additional  funding  specifically  to  achieve  higher  reliability, 
further  emphasis  is  put  on  the  practices  followed  in  our  standard  relia¬ 
bility  program.  Highlights  of  the  emphasis  are  outlined  below. 

Project  Emphasizing  Reliability 

A.  Project  Reliability  Coordinator/Staft 

B.  Emphasis  of  Same  Fundamentals  as  Standard  Program 

1.  Use  Reliable  Parts 

2.  Design  for  Reliability 

3.  Test  for  Reliability 

C.  Additional  Funding. 

The  exact  nature  of  the  reliability  emphasis  is  individually  planned  for 
each  such  project.  Emphasis  includes  provid-'  g  a  reliability  coordinator 
engineer  and  assistants,  if  necessary,  inthe  otarea.  The  coordina¬ 
tor  monitors  those  project -engineering  act-  .  ,..;s  that  primarily  affect 
reliability  and  provides  a  liaison  between  the  project  and  the  staff  groups. 
Note  that  the  coordinator  is  a  Reliability  and  Components  Group  man 
attached  to  the  project.  As  such,  he  has  a  line  of  communication  through 
the  Head  of  the  R  and  C  Group  to  the  Manager  of  R  and  D  Engineering 
in  the  event  the  project  is  aopardizing  reliability.  (See  Figure  3.)  De¬ 
tailed  duties  of  our  Reliability  Coordinators  are  shown  as  Appendix  A. 
Currently  we  have  10  c  oordinators. 
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In  the  component  parts  area  we  will  usually  use  the  highest  reliability 
parts  as  covered  by  our  company  specifications,  and  will  prepare  com¬ 
plete  specifications  for  and  qualify  special  parts.  Further,  in  a  project 
emphasizing  reliability,  additional  manhours  will  be  provided  for  de¬ 
signers  to  more  thoroughly  analyze  circuits  and  packaging.  In  the  testing 
area  a  formal  Mean  Time  Between  Failure  test  is  performed,  as  well  as 
more  exhaustive  bench  and  environmental  testing  of  equipment  models 
to  uncover  weaker  areas. 

Funds  for  these  additional  efforts  are  paid  back  many  times  by  the 
maintenance  savings  during  field  use,  as  well  as  the  added  availability 
of  the  equipment  through  increased  reliability.  Additional  funds  will  vary 
up  to  15  per  cent  of  the  development  funds. 

SOME  RELIABILITY  MEASURES 

Reliability  measures  of  our  equipment  are  collected  by  the  Reliability 
Analysis  Section  of  the  Reliability  and  Components  Group.  Such  re¬ 
liability  measures  will  serve  management  in  appraising  our  effectiveness 
in  achieving  reliability  and  are  necessary  in  developing  accurate  re¬ 
liability  estimating  techniques. 

Table  1  contains  ^liability  measures  t.  a  3  have.  Obtaining  data  for 
such  measures  is  difficult,  as  in  most  case.;  obtaining  quantitative  re¬ 
liability  measures  is  not  a  primary  objective  of  the  task  that  produced 
the  data.  However,  if  the  advent  of  contractual  reliability  requirements 
and  associated  tests  continues  to  grow,  we  will  obtain  more  such  data. 
Currently  we  have  several  contractual  reliability  requirements  and  tests 
that  will  soon  yield  more  accurate  data. 

CONCLUSIONS 

We  believe  that  our  approach  to  reliability  is  wisely  balanced.  Those 
functions  affecting  reliability  that  are  best  accomplished  by  a  centralized 
authoiity  are  delegated  to  staff  groups.  The  individual  designer  is  per¬ 
mitted  the  freedom  attain  necessary  equipment  reliability  com¬ 
mensurate  with  liis  project's  total  requirements.  Our  reliability  effort 
i.~  >asod  on  odut  at  ion  and  cooperation.  Periodic  reviews,  conducted  by 

gineering  manag  mint  and  staff  specialists,  provide  a  reasonable 
number  of  checks  upon  the  progress  of  a  project’s  reliability  efforts. 
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MTF  Measures 


In  our  approach  we  do  not  equate  reliability  individually  to  system 
design  or  component  parts  or  circuit  design  or  environmental  testing 
or  time  (MTBF)  testing  or  production.  Rather,  we  equate  reliability  to 
the  total  of  such  engineering  and  manufacturing  functions.  We  recognize 
reliability  efforts  as  a  legitimate  area  of  engineering  specialization.  Our 
Reliability  Group  performs  the  staff  functions  of  keeping  abreast  of  state 
of  the  art,  advising  management,  education,  and  consulting.  Where  re¬ 
liability  is  emphasized  on  a  project,  we  stress  the  engineering  elements 
affecting  reliability  and  have -a  Reliability  Coordinator  as  pail  of  the 
project.  ’ 

Thus  we  treat  reliability  specialization  as  a  staff  function  and  do  not 
have  separate  reliability  controlling  line  operations.  Perhaps  in  the 
furture,  the  nature  of  electronic  equipment  development  maybe  such  that 
it  must  be  controlled  in  an  analogous  manner  to  the  typical  Quality  Con¬ 
trol  exercising  surveillance  over  Manufacturing.  If  such  becomes  the 
case,  we  would  approach  this  as  total  engineering  control  and  reliability 
would  be  one  part  of  this  total  control. 

This  discussion  has  been  about  our  reliability  approach  in  the  R  and  D 
Engineering  phase.  Reliability  considerations  are  very  important  in  the 
production  and  field  use  phases.  Our  Product  Engineering  and  Quality 
Control  activities  provide  continued  attention  to  reliability  considera¬ 
tions  after  the  R  and  D  Engineering  phase. 

Motorola's  reliability  program  is  the  result  of  an  early  appreciation 
of  the  importance  of  reliability  by  our  management  and  of  many  years 
experience  in  producing  reliable  equipment.  We  believe  that  our  tested 
and  efficient  program  can  satisfy  Military  reliability  requirements. 
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RELIABILITY  COORDINATOR  DUTIES 


L  Component  Parts  Area 

A.  See  that  part  specification  and  qualification  coverage  planning 
meeting  per  DI  560/12  is  conducted, 

B.  Insure  that  optimum  component  parts  are  used  by  consultation 
with  parts  specialists. 

C.  Issue  project  standards  (AVL) 

D.  Insure  that  adequate  procurement  documents  exist. 

E.  Have  necessary  preliminary  evaluative  testing  and  qualifica¬ 
tion  to  specifications  performed. 

F.  See  that  failed  parts  are  submitted  to  R  and  C  for  analysis. 

G.  Issue  instructions  to  incoming  inspection. 

H.  See  that  parts  planned  for  final  use  are  used  in  breadboards 
at  earliest  time. 

IL  Design  Area 

A.  Have  reliability  estimate  performed. 

B.  Issue  project  derating  guide.  Follow  up  with  stress  analysis. 

C.  Issue  project  component  part  variation  guides.  Follow  up  to 
sec  if  utilized  in  circuit  design. 

D.  Issue  par*  failin'  susceptibility  guides. 

E.  See  that  thermal,  vibration,  shock,  etc.  requirements  receive 
adeq'.iat •  consideration  through  consultation  with  mechanical 

specialists. 

F.  See  that  produeibihtj.  requirements  receive  adequate  con¬ 
sideration  through  consultation  with  product  engineering/ 
quality  control  specialists. 
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RELIABILITY  COORDINATOR  DUTIES  (cont) 

Testing 

A.  Assist  in  planning  breadboard  and  prototype  performance  and 
environmental  testing. 

B.  Assist  in  planning  formal  reliability  measurement  test  and 
burn  in  when  such  are  required. 

C.  Formulate  malfunction  reporting  process  on  above  tests. 

D.  Follow  up  on  above  testing  and  malfunction  reporting. 

E.  See  that  running  time  records  are  kept  on  advanced  bread¬ 
boards  and  prototypes. 

F.  Follow  up  on  manufacturing  malfunctions. 

General 

A.  Assist  in  the  formulation  of  operational  and  environmental 
requirements  through  spec  review,  customer  contact,  etc. 

B.  Prepare  customer  required  and/or  proj.  periodic  reliability 
progress  report. 


C.  Participate  in  periodic  project  reviews,  follow  up  on. 
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RELIABILITY  AND  COMPONENTS  HANDBOOK 
MOTOROLA  INC. 

Western  Military  Electronics  Center 
Scottsdale,  Arizona 

Contents 

1.  GENERAL  AND  PHILOSOPHY 

1.1  TWENTY-SEVEN  RULES  FOR  GUIDED  MISSILE  ENGINEERS 

1.2  SOUND  ENGINEERING  -  THE  FOUNDATION  OF  EQUIPMENT 
RELIABILITY 

2.  BASIC  RELIABILITY  MATHEMATICS 

2.1  RELIABILITY  AND  TIME 

2.1.1  Failure  Rate 

2.1.2  Reliability 

2.1.3  Derivation  Notes 

2.2  MATHEMATICS  OF  RELIABILITY  MEASUREMENT 

2.2.1  Mean  Time  to  Failure  Computation 

2.3  RELL\BILITY  OF  COMBINATIONS  - 

2.3.1  Reliability  of  a  Series 

3.  DESIGN  FOR  RELIABILITY  GROUND  RULES 


Paper  on  'Triad  for  Design  Reliability" 

3.1  RECOMMENDED  APPLICATION  OF  COMPONENT  PARTS 
FOR  HIGHER  RELIABILITY 

3.1.1  Derating  of  Electrical  Component  Parts 

3.1.2  Standard  Resistance  Values 

3.1.3  Choice  of  Resistance  Values 

3.1.4  Derating  of  Electron  Tubes 
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3. 2  REALISTIC  OPERATIONAL  TOLERANCES  FOR  COMPONENT 
PARTS  AND  STATISTICAL  APPROACH  TO  CIRCUIT  DESIGN 

3. 2. 1  Variation  to  be  Expected  In  Resistors  and  Capacitors 

3. 2. 2  Variations  to  be  Expected  in  Tubes 

3. 2. 3  Variations  to  be  Expected  in  Semiconductors 

3. 2. 4  Statistical  Approach  to  Circuit  Design 

3.3  ADEQUATE  TESTING  AND  RELIABILITY  RECORDS  NEEDED 
ON  EQUIPMENT  MODELS 

3. 3. 1  Failure  Reports  on  Tubes  and  Parts 

3.3.2  Equipment  Reliability  Records  and  Reliability 

Measurement  ~  ~ 

3.3.3  Setting  of  Production  Tolerances  Based  UponMeasure- 
ments  on  a  Few  Units 


«*  cnuanoa  aid  drsfmhutzoi  or 

KZSSIia  WEAPONS  S1STVK5  HELIABIUTI 

Lt.  Col.  Henry  E.  Bodaon  sad  Otis  8,  Spoor* 

0.  S.  A rmj  Artillery  A  Missile  School,  Port  Sill 

■introduction 

The  purpose*  of  this  puper  or*  o*  follow* » 

a.  To  d*f in*  an  led**  of  reliability  for  artviiarw  missile 

•**Ua». 

b.  To  outline  a  method  of  evaluating  th*  index  of  roll  ability 
referred  to  above - 

Tbor*  la  at  proa  ant  no  uniform  system  or  theory  dealing  with  th« 
data  ruination  of  mi*all*  ayatams  roll ability.  Alaoat  every  papar  and/or 
eonfaronoa  pertaining  to  future  ad. a alia  ayotama  daala  In  aom*  aannar  with 
▼arlooa  aapaeta  of  something  call  ad  reliability.  Th*  roault  la  aoam 
confusion  relative  to  th*  doflnition  and  measurement  of  roi lability. 

Thia  papar  contain*  a  suggeeted  approach  to  th*  problems  of  doflnition 
and  vvaleation  of  tactical  roliabllity. 

Proai  th#  atandpolnt  of  tactical  artilleryman,  determination  of 
roliabllity  mast  ultimately  ba  orlantad  toward  tactical  performance, 
which  la  intimately  aaaociatad  with  the  commander’ a  ability  to  porfoim 
hia  ala sion.  Hanoa,  the  whole  approach  outlined  in  thia  paper  enoompasaes 
thoee  factor*  which  have  a  direct  bearing  on  tactical  performance,  the 
general  wiaw  ia  directed  toward  a  method  of  appraialng  the  extant  to 
which  a  given  weapon  ays tarn  meats  a  specified  set  of  standards. 

Th*  particular  factors  envisioned  as  being  significant  are  outlined 
In  Part  1  of  thia  atudy.  Generally,  the* a  factors  are  baaed  on  direct 
statement*  from  military  characteristic#.  In  one  or  two  Instances  the 
specific  factors  are  based  on  contemplated  atateeents  for  future 
military  characteristic*  formate. 

For  present  considerations,  a  pacific  factors  which  enter  the  evalua¬ 
tion  for  overall  reliability  will  be  mentioned  only  as  example*.  Gen¬ 
erally,  these  factors  will  be  referred  to  hare  categorically.  Th* 
method  presented  hare  is  not  limited  to  airy  particular  number  of 
factor*. 

The  basic  concept  under  consideration  revolves  around  th*  definition 
of  reliability,  together  with  a  specific  procedure  for  evaluation. 

Missile  weapon  systeii  reliability  la  a  measure  of  assurance  that  the 
ay* tee  will  function *as  required  under  the  operational-environmental 
condition*  and  time  limits  specified  in  the  military  characteristic*. 
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PART  I 

TACTICAL  RELIABILITY  FACTORS  FOR  MISSILE  WEAPONS  SYSTEMS 

Luring  World.  War  II -and  the  war  in  Korea,  nexd  comnanders  were  not 
gi-e&tly  concerned  with  reliability  of  weapons  systems.  On  the  battlefield, 
tie.-  had  the  utmost  confidence  that  they  could  move  their  cannons  into  a 
position,  slam  a  shell  in  the  breech,  and  fire  away  at  the  enesy.  When 
the  artilleryman  became  of  missile  age ,  he  discovered  that  there  was  a 
term  called  "countdown"  and  another  one  called  "hold."  He  soon  leaned 
that  these  words  meant  for  him  procedures  that  would  greatly  change  his 
accustomed  mode  of  operation  which  we  may  call  "charge- -shoot- -and  be 
danced."  He  began  to  hear  the  cry  "hold"  all  too  often,  and  began  to 
learn  only  too  well  that  this  meant  that  his  equipment  had  broken  down  or 
one  of  his  men  had  made  an  error,  or  both.  It  was  not  long  before  ha  be- 
(tAn  to  wonder  whether  he  could  employ  effectively  some  of  the  missile  sys¬ 
tems  that  were  being  tuned  over  to  him.  At  any  rate,  he  soon  began  to 
think  in  terms  of  reliability  or  rather  in  terms  of:  "what  are  my  chances 
for  moving  cross-country  and  arriving  at  ny  firing  position  in  an  opera¬ 
tional  status! "  He  began  to  wonder:  "What  are  ay  chances  for  checking 
out  this  missile  within  the  time  allowed  me  so  that  I  can  attack  this 
target  effectively?"  Such  thoughts  convinced  him  that  he  must  have  a 
thorough  knowledge  of  reliability  factors  in  order  to  plan  and  execute  his 
field  operations.  What  are  these  factors,  and  how  do  they  tie  in  with 
field  operations? 

The  tactical  commander  is  concerned  primarily  with  the  problem  of  ob¬ 
taining  worthwhile  effects  on  the  target.  To  get  this  result,  he  sust 
move  his  artillery  organization  into  a  firing  position  from  some  distant 
area,  establish  survey  and  communications,  and  acquire  meteorology  data, 
if  needed.  The  organization  will  have  to  emplace  and  secure  equipment, 
checkout  missiles  and  warheads,  receive  the  fire  mission,  and  launch  the 
missile.  This  sequence  of  ope rat ions  will  frequently  change  its  pattern 
because  events  on  the  battlefield  will  call  for  changes.  There  may  be 
long  periods  during  which  target  intelligence  is  being  developed,  or  in 
fact  the  targets  themselves  are  taking  form.  There  may  be  delays  in  mov¬ 
ing  the  weapons  systems  into  a  more  favorable  position  or  for  establishing 
the  maneuvering  ground  or  airborne  forces  in  an  attack  position.  There 
may  be  holds  while  waiting  for  the  target  analysis,  selection  of  appro¬ 
priate  warhead,  or  the  decision  to  fire.  It  seems  then  that  missile  field 
operations  will  frequently  be  discontinuous  affairs. 

If  these  field  operations  can  be  sep>arated  into  major  significant 
phases,  the  artilleryman  can  determine  his  chances  for  success  of  each 
phase  under  the  conditions  specified.  This  is  the  key  to  our  approach  on 
reliability  from  the  user's  standpoint;  that  is,  to  determine  the  chances 
for  success  in  each  major  phase  of  a  field  operation. 

After  the  artllj.eryman  has  acquired  the  numeric &.  figures  that  spell 
out  the  reliability  factors,  he  is  able  to  plan  and  execute  his  operations— 
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again,  probably  in  major  phase •--•o  that  ha  know*  where  ha  stand*  at  all 
times  and  parhapa  can  sake  provision*  for  improving  hla  chaacea  of  sue* 
eess.  Tor  exaapls,  if  a  battalion  co— andar  vas  to  launch  a  missile  of 
perhaps  0.7?  checkout  reliability,  he  sight  prepare  a  second  missile  and 
hare  it  on  standby  in  order  to  bolster  his  chances  for  successful  accost* 
pj.i^jsant  of  the  mission ■  In  all  of  hie  deliberations,  and  as  reflected 
In  tbs  definitions  which  follow,  the  element  of  time  is  of  prims  import¬ 
ance  .  Time  oust  be  considered  when  vs  speak  of  success  in  field  opera¬ 
tions  . 

With  this  concept  as  a  background,  we  may  proceed  to  define  rella* 
billty  ae  applied  to  th'  major  phases,  pre-flight  and  in-flight  portions 
of  the  field  operation.  at,  first,  it  is  necessary  to  define  opera¬ 
tional  status,  pre-flight  and  in-flight. 

Pre- flight  operational  status  is  the  condition  that  the  weapon 
system  (ground  equipment  and  missile)  is  capable  of  responding  to  a  fire 
mission,  or,  can  be  brought  to  such  condition  within  the  time  limits 
specified  in  the  military  characteristics. 

In-flight  operational  status  is  the  condition  that  the  missile 
(and  such  ground  equipment  as  required  to  function)  operates  during  flight 
within  the  design  parameters  as  specified  in  the  military  characteristics. 

We  can  define  reliability  with  respect  to  the  major  phases  of  pre¬ 
flight  and  in-flight  operations  as  follows: 

Pre-Flight  Operations: 

Post- movement  reliability  Is  the  probability  that  the  sys¬ 
tem  (the  ground  equipment  in  travelling  condition  and  the  complete  missile 
still  In  containers)  will  retain  pre- firing  operational  status  after. 

1.  A  ground  movement  with  organic  vehicles  over  battle¬ 
field  roads  and  cross- country  routes  of _  miles  (approximately  equal 

to  l/k  the  maximum  range  of  the  system) . 

2.  An  airlift  of _  miles  in  phase  _  of  an  air¬ 

borne  operation. 

3.  A  hellcoprter  lift  of _  miles. 

Check-out  reliability  is  the  probability  that  the  weapon 
system  will  retain  pre- firing  operational  status  during  the  period  vhen 
the  ground  equipment  is  emplaced  and  checked  out  in  firing  position  and 
a  missile  is  checked  out  and  assembled  on  the  launcher. 

Count Jctwn  reliability  is  the  probability  that  the  weapon 
system  will  retain  pre- fixing  operational  status  during  the  time  period 
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Rate  of  fire  reliability  is  the  probability  that  the  system  can 
engage  successive  targets  and  fire  missiles  at  the  cyclic  rate  prescribed 
in  the  military  characteristics. 

And,  finally,  there  is  the  general  category  of  natural  environmental 
conaiwi.oas  as  applied  to  the  functioning  of  the  system.  (It  has  been  as¬ 
sumed  in  all  of  the  previous  definitions  that  the  system  is  operating  in 
a  temperate  climate,  approximately  at  sea  level,  in  rolling,  cultivated 
terrain,  and  with  low  wind  conditions.)  Reliability  under  other  condi¬ 
tions  are  defined  as  follows: 

Environmental  reliability  is  the  probability  that  the  system  will 
retain  pre-flight  and  in-flight  operational  status  under  specified  climatic 
and  other  environmental  conditions  specified  in  che  military  characteris¬ 
tics  for: 

1.  Temperature,  humidity,  precipitation,  visibility,  fungus, 
insects,  sea  spray,  and  dust  in:  - 

a.  Cold- dry  or  cold- vet  zones. 

b.  Hot-dry  or  hot-wet  zoneB. 

2.  Terrain: 

a.  Rolling,  wooded. 

b.  Desert. 

c.  Heavy  mud,  muskeg,  or  tundra. 

d.  Mountainous  vith  little  vegetation. 

e.  Mountainous  wooded. 

3-  Barometric  pressure  equivalent  to  extremes  of  altitude  as 
specified  in  the  military  characteristics;. 

4.  Winds,  gur.tr,  and  steady,  up  to  maximum  specified  in  mili¬ 
tary  characteristics . 

The  reliability  factors  us  defined  above  should  provide  the  artil¬ 
leryman  vith  the  datu  he  needs  to  plan  and  execute  his  operations.  If 
it  Is  desired  to  evaluate  tlu:  missile  weapon  system  as  a  whole,  weight¬ 
ing  factors  should  be  applied  to  each  of  the  reliability  figures  that 
have  been  obtained.  Thene  factors  should  reflect  the  relative  import¬ 
ance  of  each  clement  of  reliability.  Tiiio  is  a  difficult  thing  to  do, 
but  we  have  looked  at  our  overall  requirements  and  have  assigned  such 


weighting  factors.  Admittedly,  these  are  arbitrary  and  they  may  change 
for  a  particular  system  or  with  changes  in  tactical  concepts.  The  weight¬ 
ing  factors  are: 


Weighting  Factors 

a.  Pre- flight  operations: 

(1)  Poet -movement  reliability 

(a)  Ground  movement  12 

(b)  Airlift  8 

(c)  Helicopter  lift  ^ 

(2)  Checkout  reliability  8 

(3)  Countdown  reliability  12 

(4)  Readiness  reliability  8 

b.  In-flight  operations 

(1)  Arming  region  reliability  8 

(2)  Buret  region  reliability  32 

(3)  Warhead  reliability  32 

c.  Durability  reliability 

-(!]  Ground  equipment  12 

(2)  Missile  12 

d.  Rate  ol  fire  reliability  3 

c.  Environmental  reliability 

(.])  Temperature ,  humidity,  precipitation, 

visibility .  fungus,  insects,  oca  spray, 
and  duct  in: 

(a)  Cold-dry  or  cold- wet  zones  2 

(>>)  Hat-dry  or  hot-wet  zones  1 
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Weighting  Factors 


(2)  Terrain 

(a)  Rolling,  vooded  1 

(b)  Desert  -  2 

(c)  Heavy  mud,  muskeg,  or  tundra  3 

(d)  Mountainous ,  with  little  vegetation  1 

(e)  Mountainous,  vooded  1 

(3)  Barometric  pressure  equivalent  to  ex-  1 

t re me a  of  altitude  in  military  charac¬ 
teristics 

(U)  Winds,  gusts,  and  steady,  up  to  maximum  1 
specified  in  military  characteristics 

The  next  portion  of  this  discussion  is  devoted  to  a  method  of  deter¬ 
mining  the  overall  reliability  of  a  missile  weapon  system  using  the 
reliability  and  weighting  factors  which  we  have  JuBt  defined . 


PART  II 

THE  DEFINITION  AND  EVALUATION  OF  AN  INDEX 
OF  KELIABILITT  FOR  MISSILE  STSTEK5 

Any  number  of  specific  contributing  factors  may  be  used.  Several 
have  already  been  defined  in  Pai  t  I  of  this  paper.  For  example.  Count¬ 
down  reliability  la  defij«ed  as  the  probability  that  the  weapon  system 
will  retain  pre-firing  operational  status  during  the  time  period  (as 
specified  in  the  military  characteristics)  from  initiation  of  a  fire 
mission  at  the  battery  to  missile  lift-off  (with  no  intervening  period 
of  standby). 

The  spociflc  factors  referred  to  above  having  been  defined,  the 
next  step  le  to  depict  the  reliability  of  each.  It  is  important  to 
note  that  each  aspect  of  system  reliability  is  defined  as  the  proba¬ 
bility  of  achieving  sore  standard  (tactically  significant  factor)  which 
has  already  been  stipulated.  These  specifications  are  usually  written, 
explicitly  or  implicitly,  in  the  military  characteristico. 

A  convenient  method  of  depicting  these  probability  numbers  is 
simply  to  plot  thorn  as  heights  of  a  probability  bar  graph.  The  proba¬ 
bility  of  achieving  a  given  specified  factor  is  the  height  of  the 
approximate  portion  (bar)  of  the  graph. 
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The  width  (baee)  of  each  bar  of  this  graph  is  proportional  to 
the  relative  importance  of  the  appropriate  tactical  factors.  Thus, 
the  width  of  each  bar  represents  a  priority. 

The  probability  dimension  of  this  graphical  plot  has  a  maxima 
length  of  1  unit.  Henoe,  the  maxima  height  of  any  particular  portion 
(bar)  of  this  graph  is  1.  If  h^  represents  the  height  of  the  i-th 
bar,  and  bi  the  base,  then  the  area  of  the  rectangle  representing  the 
relative  importance  and  reliability  of  the  i-th  factor  la 

Xi  -  bi  hj.  (1) 

where  A,  represents  th  area  of  the  rectangle.  Bach  such  area  is  called 
tha  reliability  index  for  the  particular  factor  (standard)  under  con¬ 
sideration.  It  will  be  noted  that  the  maximal*  area  of  Ai  is  b^. 

A  broken  line  through  the  mid-points  of  the  tops  of  these  rectangles 
represents  the  reliability  profile  of  the  weapon  missile  system  under 
consideration.  Figure  2  is  a  schematic  representation  of  this. 

As  many  tactical  factors  as  desired  can  be.  Included  in  the  pro¬ 
cedure  outlined  above.  Let  the  length  of  the  graphical  depiction  be 
L.  Then  the  width  (probability  dimension)  has  already  been  defined 
as  1  unit.  Hence,  e  perfect  weapon  missile  system  would  be  represented 
by 

L  x  1  «  L  square  units.  -  (2) 

The  entire  procedure  is  illustrated  by  Figure  1.  The  Index  of 
Reliability  Ir  is  defined  as  follows: 


It  will  be  observed  that  the  perfect  weapon  system,  as  noted  above, 
would  be  represented  by  L  square  units.  Hence,  the  maximum  value  of  Ir 
is  1.  Therefore. 

0  £  lr  ^  1.  (4) 

It  should  be  emphasized  that  the  evaluation  outlined  above  depends 
on  the  tactically  significant  factors  specified  in  military  characteristics 
arid  their  relative  importance  when  compared  with  each  other.  This  matter 
of  relative  importance  uniat  of  necessity  be  the  subject  of  considerable 
study  j  for  no  two  tacticians  would  rate  the  several  factors  the  same.  It 
le  not  the  purpose  of  this  paper  to  recommend  specific  weighting  factors. 

When  there  is  no  general  agreement  pertaining  to  relative  importance 
of  tactical  entities,  the  evaluation  of  Ir  can  proceed  on  the  basis  of 
the  assumption  that  all  factors  are  of  equal  importance.  Such  a  pro¬ 
cedure  should  be  used  purely  in  the  absence  of  ary  better  information. 

In  this  case,  all  base*  bp  are  equal. 


A  QUANTITATIVE  CONCEPT  OF  RELIABILITY 


PROBABILITY 


FIG.  I 


Ir  •  Index  of  Reliability 

hj.  »  Height  of  the  1-th  reliability  rectangle 

b^  *  Base  of  the  i-th  reliability  rectangle 

m  Area  of  the  i-th  reliability  rectangle 

L  ■  Length  of  the  base  of  the  entire  reliability  evaluation  sheet,  fron 
0  to  the  last  point  M. 

*  The  i-th  division  of  L,  representing  a  specified  tactically 
significant  factor. 

r(x)  =  Mathematical  function  describing  the  reliability-profile  curve. 


x  ■ 


Interval  measured  along  1  from  the  point  L  *  0. 
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RELIABILITY  PROFILE  CURVE 


PROBABILITY 


L 


FIG  2 


See  Figure  1  for  explanation  of  notation. 

Dots  on  the  broken  line  represent  eld-points  of  tops  of 
reliability  rectangles. 
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In  equation  (3)  above,  the  quantity  2  JL  li  merely  the  total  of  all 
reliability  rectangles.  It  can  also  be  regarded  as  approximately  the 
area  uixler  the  reliability-profile  curve.  Let  Ar  represent  the  area 
of  this  curve.  Then,  generally, 

L 

Ap  ■  21  «/  r(x)  dx.  (5) 

10 


It  is  readily  seen  that  the  maxima  value  of  Ap  is  L.  Hence, 

0  ■  Aj,  ■  L,  (6) 

The  concepts  and  procedures  outlined  herein  represent  an  approach 
to  the  problem  of  overall  evaluation  of  aisslle  systems  reliability. 

There  is  a  need  for  standardisation  with  respect  to  the  following! 

(1)  The  number  and  nature  of  tactically  significant  entities 
to  be  considered. 

(2)  The  relative  importance  of  these  tactical  factors. 

There  will  ordinarily  be  a  minimum  height  for  any  particular  h^. 
Below  this  minimum,  the  value  of  Aj.  will  assume  the  value  0  by  definition. 
The  Reliability  Index  is  one  measure  of  reliability.  It  is  not  a  direct, 
compound  probability  of  accomplishing  a  tactical  mission,  but  is  a 
measure  related  to  such  a  probability.  Each  M.  is  a  compound  probability, 
and  is  calculated  by  the  tactical  troops  from  lata  collected  during  field 
operations  and  firings.  It  is  a  product  of  subsidiary  probabilities. 

An  alternative  procedure  to  that  outlined  above  would  be  to  multiply 
the  probabilities  Let  1^  denote  this  product.  Then, 

Im  *  n  Mi.  (?) 

Hence,  a  useful  concept  is  one  which  encompasses  tooth  Ir  and  1^ . 

This  measure  is  called  IP,  and  is  defined  as  follows: 


'  Ir  *  *m 
F  2 

Since  the  maximum  of  Ir  or  IB  is  1,  the  maxim, un  value  of  Ir  is  also 
1.  By  definition,  the  -value  of  1^  is  0  when  gither  I, 


(6) 


rr  hr  1^  becomes  0. 


M 
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♦I.BT'S  DEFILE  OUR  COMPONENT  RELIABILITY  GOALS" 

R .  J .  Nordlund 

Technical  Director 

Weapons  Guidance  Laboratory,  WADD 

This  paper  is  intended  as  a  study  of  the  basic  nature  of  the 
reliability  problem  concerning  the  "building  blocks*  for  missiles  and 
space  vehicles.  The  intent  is  to  approach  the  problem  of  basic  goal 
definition  from  the  standpoint  of  our  broad  national  interests  and  to 
examine  the  road-blocks  in  the  way  of  achieving  the  goal.  Although 
the  problems  are  equally  serious  and  difficult  in  the  mechanical  and 
the  electro-mechanical  areas,  this  discussion  is  intended  primarily 
for  the  various  managements  which  comprise  the  electronics  industry  of 
our  nation  for  reasons  which  will  become  evident  as  we  read  on. 

It  has  been  a  matter  of  great  concern  to. the  author  since  the  end 
of  World  War  II  that  the  quality  of  the  component  parte  which  must  form 
the  heart  of  ever-increasingly  complex  electronic  instrunentation  is 
completely  inadequate  even  for  present-day  requirements.  It  is 
encouraging  that  some  leaders  in  the  industry  are  now  becoming  aware  of 
the  gravity  of  the  situation  as  evidenced  by  statements  published  in 
the  March  1959  issue  of  "Signal  Magazine 

The  designer  of  complex  systems  today  is  in  much  the  same  situa¬ 
tion  as  an  architect  would  find  himself  if  he^were  connissioned  to 
design  a  skyscraper  to  outdo  the  Empire  State  Building,  and  then  dis¬ 
covered  that  his  best  building  material  was  adobe  blocks, 

Why  is  the  vital  role  of  basic  parts  not  evident  to  usf  As 
Paul  Darnell  said  in  the  "Signal,*^  "It  should  be  self-evident.*  Is  it 
because  a  resistor  tube,  capacitor,  valve,  relay  or  tubing  Joint  is  not 
a  glamorous  thing  to  a  designer?  This  should  not  be  of  concern  to 
creative  designers  who  can  visualize  the  completed  device,  I  am 
reminded  of  two  bricklayers  I  encountered  one  spring  evening  in 
Philadelphia  as  I  was  taking  my  customary  evening  walk.  These  men  ware 
working  overtime  in  a  large  new  church.  The  first  workman  was  doing  a 
sloppy  job,  spilling  mortar  over  himself  and  the  ground.  Just  for  fun 
I  asked  him  what  he  was  doing  and  he  grunted,  "Laying  bricks,  stupid," 
Thus  suitably  rebuffed  I  walked  on  a  quarter  of  a  block  where  the 
second  man  was  working,  and  was  impressed  by  the  contrast.  This  lad, 
a  tall,  lanky,  happy-looking  colored  lad,  was  doing  an  extremely  neat 

1.  Page  6,  Signal  (Journal  of  the  Armed  Forces  Conmunicatlons  & 
Electronics  Association), 

2 .  Op .  cit .  p.  6 
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job.  I  repeated  my  question,  end  this  boy  straightened  up  from  his 
work,  looked  up  into  the  sky  and  said,  *Suh,  I'se  building  a  beautiful 
cathedral.* 

Or  is  it  that  we,  as  scientists  and  engineers,  have  become 
enamoured  of  the  mere  achieving  of  performance,  and  feel  that  we  have 
achieved  the  final  goal  if  we  prove  the  feasibility  of  an  idea  and 
have  made  something  that  works,  even  if  it  is  yet  far  from  satisfactory 
for  the  required  application? 

Or  have  we  as  a  nation  been  so  busy  becoming  prosperous  during 

the  past  14  years  that  we  have  accepted  lack  of  progress  in  our  basic 

building  block  quality  as  a  normal  thing? 

I  do  know  one  thing  to  be  true,  and  that  is  that  we  have  written 

and  spoken  enough  millions  of  words  in  the  past  ten  years  on  the  un¬ 

reliability  of  component  parts.  It's  high  time  we  sit  down  and  define 
whet  our  goal  must  be,  and  then  go  about  the  job  of  achieving  that  goal. 
I  have  no  quarrel  with  most  of  what  I've  heard  or  read,  but  I  feel  that 
our  thinking  has  become  stereotyped,  and  sterile,  and  we  either  do  not 
mean  what  we  say,  or  have  failed  to  put  our  convictions  into  action. 

This  paper  was  conceived  one  sleepless  night  on  an  airplane,  when  I 
tried  to  put  myself  to  sleep  by  reading  the  previously  mentioned  issue 
of  "Signal.*  This  particularly  well  done  issue  was  dedicated  to 
•components"  and  I  read  it  from  cover  to  cover.  I  shall  hereafter 
refer  to  this  magazine  as  *the  book."  In  his  article  in  that  •book,* 

Dr.  Homer  J.  Stewart'  said,  in  speaking  of  the  NASA  program.  *The 
guidance  and  communication  equipment  must  be  tailored  to  suit  the 
special  needs  of  space  missions.*  What  did  he  mean  by  this,  especially 
the  •special  needs"  part?  Let  u3  examine  a  specific  example  of  such 
•special  need."  Preliminary  studies  indicate  that  a  satellite  elec¬ 
tronic  system,  to  be  economical  and  generally  suitable,  should  have  a 
95%  certainty  of  three  years'  mean-time-between  failure.  If  the  system 
contains  10,000  component  parts,  and  I  know  it  will  if  we  build  It 
with  present  teclmiqucs,  such  parts  must  have  a  failure  rate  for  each 
part  of  approximately  2  x  10“^  or  0.0002  failure  per  million  hours. 

This  is  a  mean-timc-between-failure  for  each  part  of  about  5°8*000*00° 
hours,  and  this  i3  58.000  yearsl  Fantastic,  isn't  it?  Impossible 
too,  we  might  say,  and  ait  down  and  examine  why  we  want  to  do  such  a 
thing,  oft or  all.  Obviously,  many  people  ore  in  the  frame  of  mind  to 
thus  sit  down  on  the  problem,  judging  from  the  extreme  difficulty  of 
applying  resources  to  the  basic  problem  during  the  past  few  years. 

I  jut.  sure  that  you  will  all  agree  that  space  has  suddenly  become 
our  new  frontier,  one  that  provides  a  challcngo  to  all  nations,  more 
so  than  at  any  other  period  in  history.  The  only  comparable  period 
was  the  opening  up  of  the  no*  world  for  exploration  and  conquest 

3  •  Op  .c  i  t  .p  .  1'9 
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following  Chri'topher  Columbus.  Now,  as  in  that  time,  the  prestige 
and  power  of  nations  will  be  determined  to  a  large  extent  by 
successes  in  this  new  frontier.  The  fact  that  we  falter  and  temporize 
is  a  serious  indictment  of  our  long  range  foresight  and  planning 
ability.  We  must  not  allow  profits  and  the  enjoyment  of  the  good  life 
to  blind  us  to  the  impact  of  the  future  or  dull  our  zeal  to  achieve 
this  so-called  impossible  capability.  The  fact  that  we  concentrate  on 
small-step  gains  is  indicative  of  our  small  thinking.  I  am  fully 
aware  that  great  gains  are  being  made  to  solve  our  present  day-to-day 
reliability  problems.  Many  companies  are  doing  this  right;  now.  How¬ 
ever,  I  feel  .that  our  major  resources  must  be  .applied  to  research  on 
the  molecular  structure  of  the  materials  used  to  perform  electronic 
functions  to  make  possible  the  solution  of  the  "impossible*  reliability 
problem.  I  say,  "Let's  make  the  foot  soldier  carry  a  heavier  load  of 
conmunications  gear  for  a  few  more  months  or  even  years,  while  we 
attack  the  research  problems  whose  solution  will  provide  adequate 
space  vehicle  components." 

I  read  somewhere  else  in  "the  book"  in  an  article  on  the  great 
benefits  that  accrue  from  miniaturization  that  "in  addition  to  these 
size,  weight  and  efficiency  factors,  it  is  important  to  realize 
that  micro-module  construction  will  improve  the  reliability  of  Space 
Age  electronic  systems «•  Here  we  have  it  again  -  the  relegation  of 
reliability  to  a  by-product  status.  This  is  significant  because  it 
represents  our  thinking,  that -ehort-term  performance  and  product 
design  are  the  things  we  are  striving  for,  and  that  "isn't  it  nice 
if  we  also  can  happen  to  improve  reliability  at  the  same  time?" 

Further  along  in  the  article  the  same  author  writes,  ■-  -  -  -  This 
must  continue  at  an  accelerated  pace,  for  our  nation's  military 
techniques  depend  on  our  ability  to  stay  ahead  in  electronics."  He 
did  not  say  what  or  who  we  must  3tay  ahead  of.  I  insist  again  that 
as  far  as  electronics  in  the  Space  Age  i3  concerned,  we  are  not 
ahead  of  but  seriously  behind  present  requirements.  The  usefulness 
of  orbiting  and  space  vehicles  will  be  seriously  degraded  far  beyond 
the  point  of  no-return  if  we  can  not  make  ordera-of -magnitude  im¬ 
provement  in  reliability  of  complex  comnunications  and  guidance 
systems.  I  hope  everyone  in  this  meeting  will  agree  that  this  kind 
of  improvement  cannot  be  obtained  by  small  by-product  improvements 
in  the  present  component  programs.  If  you  do  so  agree,  we  are  ready 
to  define  what  our  goal  must  be  in  component  reliability# 

THE  GOAui  To  place  reliability  at  the  top  rung  of  the  priority 
ladder  in  applying  resources  to  our  basic  and  applied  research  pro¬ 
grams  and  then  to  direct  these  resources  toward  achieving  step- 
function  breakthrough  in  the  use  of  the  molecular  structure  of  matter 
as  it  relates  to  Ion- -life  performance  of  the  various  electronic 
functions  in  compl -  tely  new  ways. 
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The  establisliment  of  a  goal  ia  no  better  than  the  prov.rtiol  N 
Year's  resolution  unless  we  i..jnediately  turn  our  attention  to  the 
road-blocks  in  the  way  of  its  achievement.  Let's  now  look  at  some  of 
these« 

1.  IS  THE  ELECTRONICS  INLUbTP.Y  CGi'CEr.KXL.  -ITH  THE  SP/Xf.  AGx.? 

If  you  agreed  with  wnat  I  said  earlier  regarding  space  as  the  new 
world  frontier,  the  answer  can  only  be*  YxSl  Since  this  will  be  the 
first  involvement  of  this  fast-growing  giant  with  a  frontier  in  which 
the  nation's  future  position  in  the  world  will  be  determined,  this  is 
indeed  a  momentous  conclusion.  The  revolution  in  propulsive  powor 
development  for  flight  has  caught  the  electronics  industry  up  in  its 
whirlwind,  and  the  two  must  now  be  considered  as  co-equal  partners  in 
this  technological  frontier,  even  though  there  may  be  reluctance  on 
the  parv  of  some  elements.  History  shows  that  pioneering  In  a  new 
frontier  is  rarely  popular  with  the  masses  of  people  involved  there¬ 
with.  The  first  recorded  pioneering  was  accomplished  by  the  Israel¬ 
ites  under  Moses.  When  life  became  complicated,  they  reacted  thus, 
•What  have  you  done  to  us,  in  bringing  us  out  of  Egypt?  Ia  this  not 
what  we  said  to  you  in  Egypt,  'Let  us  alone  and  let  us  serve  the 
Egyptians'?*  The  second  one  of  note  was  the  western  frontier 
development  out  of  which  grew  the  United  States  of  America.  Here 
several  nations  were  in  competition,  but  success  went  to  our  new 
nation  because  we  possessed  the  men  of  wisdom  and  foresight,  and  the 
technological  courage  to  accomplish  their  goals.  Here  again  there 
were  many  who  looked  back  with  longing  to  the  good  old  days  left 
behind,  or  grumbled  fiercely  that  the  whole  thing  was  not  worth  the 
effort,  and  turned  back.  The  men  who  had  the  courage  to  risk  capital 
and  effort  grew  eminently  successful  (and  rich)  and  with  them  ro3e 
the  fortunes  of  the  nation.  I  believe  the  same  thing  applies  in  our 
new  frontier  in  regard  to  our  components  reliability  situation. 

2.  SPACE  HOT  TO  BE  U3£D  BY  THE  MILITARY ? 

The  opening  sentence  of  tho  National  Aeronautics  and  Space  Act  states, 
and  I  quote,  •-  -  -  -  it  is  the  policy  of  the  United  States  that  act¬ 
ivities  in  space  should  be  devoted  to  peaceful  purposes  for  the  bene¬ 
fit  of  all  mankind. •  None  of  us  disagrees  in  principle  with  this 
admirable  policy.  However,  I  believe  it  is  extremely  difficult  to 
distinguish  between  military  and  non-military.  In  fact,  I  believe 
that  it  is  a  mistake  to  insist  on  a  differentiation  when  we  are  in  a 
•frontier  situation"  undor  conditions  of  'inactive*  conflict.  Brig. 
Gen.  Homer  A.  Boushey  covered  this  difficult  problem  in  admirable 
fashion  in  an  address  to  the  American  socket  Society  entitled,  *Air 
Force  Uses  of  Spuce"^  last  17  November,  It  is  a  refreshing 
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and  challenging  discussion  of  philosophy  regarding  "military*  and 
"peaceful*  operations.  In  the  technical  sense,  it  is  gonerally  true 
that  research  and  development  of  military  systems  requires  n  much 
higher  degree  of  excellence  and  quality  in  basic  components  than  is 
•-  s’Tnercially  available,  and  that  people  engaged  in  this  kind  of 
technical  work  tend  to  consider  in  much  greater  detail  the  ferocity 
of  the  environment  in  which  equipment  must  operate  than  do  those  who 
are  engaged  in  ’scientific  applications*  work.  These  considorctions 
are  directly  applicable  to  the  problem  of  reliability.  «fe  must 
face  up  to  the  fact  that  unreliability  of  basic  components,  right 
now,  is  the  greatest  deterrent  to  extension  of  technlcuea  into  the 
space  frontier.  Here  we  cannot  draw  very  much  or.  history,  because 
of  the  revolutionary  nature  of  our  approach  to  thia  frontier,  exoept 
to  note  that  success  in  past- frontiers  was  due  in  large  measure  to 
the  willingness  of  key  men  to  eccept  bold  new  ideas  and  techniques 
wherever  they  arose,  with  no  argumont  as  to  whether  or  not  there 
were  military  considerations  to  worry  about  in  a  ‘peaceful*  era. 

3.  THE  PIiU.050PHj.CaL  ATTIIUDE  -  *VKf  BOTHER?* 

This  attitude  has  no  place  among  those  who  would  probe  the 
frontiers.  The  best  example  of  this  attitude  was  cited  by  General 
O'Connell  in  *the  book’5  in  a  story  about  the  great  naturalist, 
Thoreau.  Thorcau  had  a  friend  who  was  involved  with  the  linking  of 
Maine  with  Texas  by  new  telegraph  lines  and  who  expressed  his  pride 
in  this  accomplishment,  to  Thoreau.  Thoreau  completely  deflated  his 
friend  when  he  thoughtfully  said,  "What  in  heaven's  name  can  the 
people  in  Maine  have  to  say  to  Texans?*  This  philosophy  is  very 
prevalent  in  our  country  today.  There  are  those  who  say,  *1<#iy  in 
heaven's  name  do  you  want  a  satellite  relay  that  can  operate  for 
three  years?*  and  they  express  themselves  by  withholding  funds  for, 
achieving  the  basic  development  which  would  make  this  possible. 

These  are  the  same  type  of  people  who  said  cloven  years  ago,  * Vhy 
in  heaven's  name  do  you  want  inertial  reference  equipment,  and 
complex  computers  in  packages  small  enough  for  a  flight  vehicle?* 
Getting  around  these  obstacles  is  a  formidable  management  problem, 

k.  THE  SOLUTION  15  ’MAKE  IT  5IMPLE* 

w'e  do  not.  need  to  spend  much  time  with  this  road-block,  but  wo  do 
still  hear  it.  Phis  is  the  age-old  yearning  to  return  to  the  simple 
problems  of  the  "good  old  days."  This  is  the  expression  of  those 
caught  up  in  *frontiering"  against  their  will  or  inclination.  Our 
present  frontier  presents  technical  problems  which  are  not  simple, 
and  it  i3  folly  to  waste  time  in  hoping  that  the  solution  can  be  a 
simple  one. 

5.  signal  (Journal  of  the  Anr.-d  Fo.ces  Coamunicat ions  &  Electronics 

Ar.sori  >tion)  p.  pi 


5.  RELATIOIS  OF  RaTUIIaLS  TO  COMPONENTS  HOT  UNDERSTOOD 

This  statement,  of  course,  applies  in  general,  but  I  am  most  con¬ 
cerned  with  this  relationship  in  electronic  components  because  I 
■\  2  no  general  awareness  of  the  need  to  understand  the  molecular 
structure  of  the  materials  used  for  such  components.  On  the  other 
hand,  I  know  that  there  exists,  at  least  within  the  Air  Force,  a 
potent  program  of  materials  research  to  speed  the  availability  of 
reliability  in  other  types  of  components.  Published  statements 
have  indicated  some  of  the  problems  that  exist  in  this  regard,  chief 
of  which  is  the  matter  of  who  should  pay  for  such  basic  work.  Some 
spokesmen  for  the  components  industry  have  said  that  government  muat 
pay  for  the  basic  research  and  development  program.  This  indicates 
that  it  is  felt  tliat  the  requirement  is  only  a  military  or  defense 
problem,  which  is  true  in  a  narrow  sense.  However,  in  the  broad 
sense,  if  the  electronics  industry  is  to  assume  its  role  as  co¬ 
partner  with  the  propulsion  industry  in  the  3pace  frontier,  then  the 
problem  becomes  one  of  identification  with  the  future  strength  of 
our  nation,  and  '.ence,  a  problem  of  solution  for  the  conmon  good. 

I  feel  that  our  economic  history  proves  thut  the  financial  rewards  to 
those  who  have  taken  up  such  challenges  in  the  past  have  always  been 
substantial.  These  rewards  have  been  so  substantial  in  many  cases  in 
the  past  that  they  have  been  cause  for  alarm  on  the  part  of  our 
government.  Obviously  this  discussion  is  most  pertinent  to  the  large 
companies  who  are  leaders  in  the  industry,  because  only  they  possess 
the  resources  necessary  to  adequately  attack  such  an  expensive  and 
difficult  endeavor. 

6.  PRODUCTS  VS  QUALITY 

I  have  previously  alluded  briefly  to  the  tendoncy  we  have,  as  a 
nation,  to  concentrate  most  of  our  effort  on^new  applications  made 
possible  by  a  major  research  breakthrough  and  live  with  any  quality 
which  happens  to  evolve.  Mr.  P.  E.  Haggerty,  writing  in  the  March 
issue  of  "Signal"  said,  "The  transistor  is  important,  not  just 
because  of  its  own  amaoing  properties  and  what  they  mean  to  elec¬ 
tronics,  but  because  it  was  the  big  breakthrough  the  industry  needed 
to  ma%e  it  focus  on  a  new  frontier.  This  new  frontier  is  the  very 
structure  of  matter  itself.  -  -  The  I-feser  3nd  the  parametric 
amplifier  already  have  resulted,  and  it  is  safe  to  predict  these  tire 
just  the  beginning.  As  our  insight  into  the  solid  state  deepens,  we 
will  be  able  to  construct  true  solid  state  circuits  performing 
complete  electronic  functions  and  obsoleting  our  concepts  of  com¬ 
ponents  such  as  the  transistor,  diode,  resistor,  capacitor,  or 
inductance  as  »c  hold  them  today.  Solid  state  circuits  promise  size 
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reductions  of  1000  to  1  from  present  printed  circuit  concepts,  large 
reductions  from  present  concepts  of  miniature  module  circuits,  and 
almost  inmeasurable  improvement. in  reliability  It  is  highly 

gratifying  to  hear  this  from  leaders  in  the  industry.  There  is  no 
r-.-v  precious  commodity  in  our  nation  today  than  analytical  thinking 
on  the  part  of  management,  coupled  with  foresight  and  planning 
ability  based  on  a  technical  foundation  of  know-how  as  to  what  is 
important,  I  also  have  complete  faith  in  the  ability  of  our  scien¬ 
tific  comnunity  to  achieve  tremendous  things  if  given  the  resources 
and  the  encouragement  which  they  need,  I  feel  that  in  the  pa9t  a  pre¬ 
ponderance  of  emphasis  has  been  placed  on  making  •things.*  That 
these  ’things*  are  commercially  successful  is  not  a  final  answer#  I 
believe  wa  have  reached  a  point  of  decision  on  this.  I  don*t  think 
that  there  is  much  doubt  that  unreliability  has  been  the  main  stumbling 
block  in  sales  of  color  television  receivers.  Is  this  the  order  of 
complexity  that  can  be  tolerated  with  the  present  level  of  components, 
even  conmercially? 

Drastic  steps  must  be  taken  by  both  government  and  industry  to 
achieve  the  reliability  goal  we  have  set.  A  start  has  been  made  by 
the  Department  of  Defense  through  its  Ad  Hoc  Study  Group  on  Parts 
Specifications  Management  for  Reliability,  Many  of  you  have  heard  a 
description  of  this  effort.  The  work  of  this  Ad  H0c  Group  uncovered 
uany  problems  which  government  must  solve,  particularly  in  regard  to 
specification  control  of  basic  parts  quality  and  to  procurement  and 
operational  practices.  One  especially  acute  problem  in  regard  to 
procurement  practices  lies  in  the  resupply  of  components  for  field 
replacement.  Many  forces  have  been  at  work  to  make  competitive 
pricing  the  only  ba3is  of  selection  of  resupply  sources.  This,  of 
course,  precludes  any  hope  of  maintaining  a  high  quality  level  of 
these  parts  -  in  fact  forces  a  situation  in  which  absolutely  inade¬ 
quate  parts  arc  purchased  for  replacement  of  field  failures.  Solu¬ 
tions  to  these  wrong  practices  must  be  found  to  make  present 
equipment  useable.  This  is  but  one  example  of  the  many  things  which 
must  be  corrected  simultaneously  with  immediate  action  to  achieve  the 
goal  described  herein. 

In  conclusion,  both  Industry  and  Government  must* 

1,  Have  the  courage  to  admit  that  our  present  component 
reliability  situation  is  indeed  serious, 

L.  Kivo  the  /-cal  to  achieve  step-function  advances  by 

liberal  support  and  ty  encouragement  of  esoteric  ideas. 

3.  Develop  a  sense  of  proportion  which  will  avoid  the 

expenditures  of  billions  for  system  development  and  next 
tu  nothing,  toward  achieving  the  component  reliability 
which  will  make  the  systems  worthwhile. 
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It  is  significant  that  the  achievement  of  building  components 
of  quality  which  made  the  ilnpire  l,tate  Building  possible  was  attained 
by  learning  the  molecular  structure  of  concretes  and  steels  and 
developing  the  required  strength  and  durability  by  careful  control 
of  <-hat  molecular  structure.  Our  goal  is  clean  We  must  achieve 
this  kind  of  durability  and  reliability  in  missile  and  space  vehicle 
components. 


A  UNIQUE  AP  ROACH  TO  RELIABILITY  DESIGN  REVIEWS 


R.  S.  Cazanjlan 

Sylvania  Electric  Products  Inc. 
Mountain  View,  California 


Introduction 


The  broad  objective  of  our  design  review  effort  is  to 
create  early  project  awareness  and  final  system  compliance 
with  quantitative  military  requirements.  The  primary  chal¬ 
lenge  in  this  area  has  become  the  translation  of  this  general 
objective  into  specific  task  activities.  In  other  words,  it 
has  become  necessary  to  clearly  define  the  responsibilities 
of  cognizant  individuals  in  the  program.  As  a  result  of 
experience  acquired  on  several  projects,  it  became  apparent 
that  an  integrated,  comprehensive  manual  was  required  as 
the  basic  vehicle  for  implementing  this  activity. 

Early  Project  Experiences 

The  initial  manual  which  was  developed  attempted  to  pro¬ 
vide  the  requisite  components  for  a  successful  design  review 
program.  The  organizational  relationships  'mong  engln.  oring 
personnel  were  defined  in  great  detail. 

Throughout  the  first  phases  of  the  project  an  attempt 
was  made  by  reliability  engineers  to  review  designs  entirely 
segregated  from  development  personnel.  This  type  of  opera¬ 
tion  proved  unsuccessful  due  to  the  lack  of  background  infor¬ 
mation  which  the  development  engineer  possessed,  but  which 
was  not  obvious  on  the  detailed  drawings  under  study.  It  be¬ 
came  obvious  that  a  well-integrated  plan  of  close  liaison 
among  cognizant  personnel  was  necessary.  As  a  result,  the 
conference  approach  for  design  review_  was  chosen. 

Initially,  conference  meetings  were  time  consuming  and 
burdensome.  It  was  found  that  due  to  the  lack  of  communi¬ 
cation  prior  to  the  conference,  discussions  centered  about  a 
technical  understanding  of  the  circuit  or  package  involved 
rather  than  a  comprehensive  design  review.  As  a  result,  one 
major  deficiency  consistently  prevailed;  the  development 
engineer  did  not  have  a  clear  concept  of  the  areas  in  which 
the  committee  could  contribute  to  his  design.  He  was  con¬ 
fused  as  to  the  real  purpose  of  the  meeting.  His  attitude 
could  be  summarized  by  two  questions,  "What  can  this  group 
contribute  that  I  have  not  already  considered?"  "If  they 
really  have  the  ability  to  inject  reliability  into  designs, 
why  can't  it  be  expressed  in  specific  terms  and  made 
available  to  me  aa  a  design  guide  earlier  in  the  project?" 

The  answers  to  these  questions  have  become  the  basis  of  our 
latent  approach  to  design  reviews. 
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Current  Design  Review  Procedures 


The  "present  design  review  activity,  in  connection  with 
one  of  our  major  projects,  incorporates  what  we  consider  to 
h"  a  unique  approach.  In  order  to  successfully  implement 
the  effort,  we  have  developed  a  new,  comprehensive  manual. 

This  document,  in  addition  to  listing  organizational  require¬ 
ments,  provides  detailed  design  checklists  for  use  by  devel¬ 
opment  engineers  throughout  the  entire  development  cycle, 
from  initial  specifications  to  ultimate  system  evaluation. 

In  other  words,  we  have  attempted  to  itemize  in  a  logical 
manner  all  the  necessary  ingredients  for  developing  a  re¬ 
liable  desigr.  The  integration  of  the  design  review  effort 
with  over-all  development  has  been  attempted  by  time-phasing 
reviews  with  established  program  milestones. 

Item  1  of  the  Appendix  illustrates  the  general  areas  for 
which  reviews  are  scheduled.  Notice  the  variance  in  the 
scope  of  meetings  versus  time.  As  can  be  seen,  design  re¬ 
views  should  be  started  early  in  the  program  on  a  broad 
general  level.  When  development  approaches  the  hardware 
phase,  detailed  meetings  of  reduced  scope  are  required. 
Finally,  as  system  integration  takes  place,  the  area  of  inter¬ 
est  again  broadens.  There  is  one  common  pitfall  associated 
with  the  above  scope  variations.  The  scope  of  the  meeting 
must  not  exceed  its  intended  purpose.  In  other  words,  wo 
should  not  attempt  to  resolve  all  the  shortcomings  of  the 
project  in  one  all-encompassing  discussion.  Ir  addition, 
only  the  required  personnel  must  be  brought  into  each  con¬ 
ference.  For  example,  a  high  level  project  engineer  might 
not  be  necessary  In  a  discussion  of  a  particular  card  hold¬ 
down.  However,  his  services  are  mandatory  in  a  meeting  which 
establishes  and  Interprets  specification  requirements.  It 
should  be  the  responsibility  of  an  appointed  design  review 
coordinator  or  meeting  chairman  to  insure  that  cognizant  per¬ 
sonnel  are  informed  and  prepared  prior  to  each  meeting. 

At  this  point,  let  U3  briefly  discuss  the  various 
sections  and  checklists  which  have  been  devised  for  the  devel¬ 
opment  cycle.  In  this  manner,  we  can  further  illustrate  the 
nature  of  th_  items  that  ve  feel  are  necessary  for  a  com¬ 
prehensive  design  review. 


Logic 


.‘Ctrical 


The  electrical  logic  review  Is  performed  prior  to 
the  preliminary  layout  of  circuitry.  During  this 
meeting,  pertinent  checklist  questions  are  discussed 
briefly.  Specific  personnel  are  called  upon  to  provide 
the  basic  answers .  However,  all  attendees,  having 
developed  a  familiarity  with  the  checklist,  are  aware 
of  the  general  nature  of  the  conference  material 
scheduled  tor  review. 
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The  intent  of  this  conference  is  to  review  required 
system  effectiveness  and  investigate  possible  specifi¬ 
cation  problem  areas.  Such  things  as  the  feasibility  of 
logical  system  functions  and  reliability  requirements 
come  under  close  scrutiny. 

If  unresolved  problem  areas  remain  at  the  conclusion 
of  the  meeting,  action  items  are  assigned  to  attendees 
and  a  follow-up  conference  is  scheduled.  Some  of  the 
checklist  questions  for  this  meeting  are  included  in 
Appendix  Item  2. 


I.ogic  Reviews  -  Physical  Concept 

The  physical  concept  logic  review  is  performed 
after  the  electrical  review  but  prior  to  preliminary  lay¬ 
outs.  The  Intent  of  this  meeting  is  to  review  physical 
design  proposals  for  size,  weight,  density  of  packaging, 
cooling  techniques,  and  mechanical  layout.  Prior  to  the 
meeting,  it  is  necessary  for  attendees  to  familiarize 
themselves  with  physical  environments,  vehicles  and 
available  space,  and  tentative  system  block  diagrams. 

During  the  meeting,  the  group  reviews  areas  such  as 
proposed  module  sizes,  card  hold-down  techniques,  and 
RFI  requirements.  Some  of  the  considerations  included 
on  the  checklist  are  shown  in  Appendix  Item  3. 

Functional  Circuit  Reviews 

The  functional  circuit  review  13  conducted  after  the 
preliminary  drawing-board  layout  but  prior  to  hardware 
fabrication.  The  purpose  of  this  meeting  Is  to  review, 
for  the  first  time,  actual  circuitry  with  regard  to  feasi¬ 
bility  and  reliability.  The  development  engineer,  who  has 
used  the  checklist  in  developing  his  design.  Is  prepared 
to  discuss  such  things  as  input  and  output  requirements, 
interface  problems,  and  effects  of- failures  upon  functional 
capabilities.  At  the  completion  of  this  phase,  the  origi¬ 
nal  logic  diagram  should  emerge  complete  and  substantially 
verified.  Appendix  Item  4  contains  some  selected  questions 
from  this  checklist. 

Part  and  Mo lulo  Reviews 

Part  and  module  reviews  are  scheduled  after  prelimi¬ 
nary  layouts  during  the  "haywire"  breadboard  stage.  All 
questions  pertaining  to  module  configurations  such  as 
hot  3 pots ,  heat  generators,  and  vibration  sensitive  parts 
must  bo  resolved  at  this  time,  prior  to  detailed  drf’ings. 
The  module,  defined  In  this  paper  as  the  next  lower  vel 
of  assembly  from  a  line  replaceable  unit,  mu3t  be  checked 
with  rorard  to  the  proper  application  of  parts,  coolant 


adequacy,  and  marginal  operation  versus  environmental 
combinations.  During  the  meeting,  the  group  discusses 
the  engineers'  sketches,  -test  reports,- reliability  pre¬ 
dictions,  and  calculations.  Once  again  all  personnel 
attend  the  conference  with  full  knowledge  of  the  check¬ 
list  which  will  be  used  to  establish  conformance  with 
reliable  design  practices.  Item  5  of  the  Appendix 
lists  selected  questions  from  this  list. 


Line  Replaceable  Unit  Review 


After  the  module  design  has  been  established  and  pre¬ 
liminary  LRU  design  initiated,  the  unit  reviews  are  imple¬ 
mented  and  continued  until  design  is  complete.  The  in¬ 
tents  of  these  meetings  are  quite  similar  to  those  indi¬ 
cated  for  the  part  and  module  reviews.  However,  the 
scope  of  material  increases  to  include  larger  segments  of 
the  hardware.  Greater  emphasis  is  placed  on  packaging, 
environment,  cost,  ease  of  maintenance,  and  fabrication. 
This  particular  phase  of  development  becomes  the  last 
major  hardware  effort.  Outstanding  problems  concerning 
physical  design  should  be  resolved  at  this  point.  Item  6 
of  the  Appendix  lists  some  questions  from  the  LRU  review 
checklist . 


System  Worth  Reviews 


A3  soon  as  completed  line  replaceable  unit  designs 
are  available,  the  final  logical  marriage  reviews  are 
implemented.  It  is  the  purpose  of  these  meetings  to 
assure  system  capability,  effectiveness,  .d  over-all 
worth.  Final  specifications,  block  di~  ,  us,  and  system 
test  results  are  reviewed  for  adcquac  checklist 

questions  of  Appendix  Item  7  are  inclu  -o  further 
illustrate  this  area. 


Design  Review  Pitfalls 


As  mentioned  previously,  the  greatest  pitfall  in  design 
reviews  has  been  the  cver-anbitlous  objectives  of  the  meetings. 
Doth  economic  and  engineering  limitations  have  necessitated 
compromise  and  redirection.  We  cannot  expect  the  design  to  bp 
fully  discussed  at  these  conferences.  The  prime  purpose  of  a 
formal  design  review  meeting  must  be  to  insure  that  an  adequate 
effort  has  been  made  by  the  designer.  This  is  evidenced  by 
the  calculations  and  reports  required  by  the  checklists.  In 
addition,  where  particular  problem  areas  develop,  the  design 
review  group  must  insure  follow-up  and  resolution.  It  would 
hardly  be  feasible  to  expect  a  group  to  review  in  detail  the 
great  number  of  considerations  involved  in  development.  For 
thl3  reason,  our  manual,  composed  of  some  '*20  checklist 
questions,  was  designed  to  be  self-contained.  In  addition,  - 
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an  appendix,  containing  approximately  100  detailed  design  pro¬ 
cedures  (see  Appendix  Item  8), was  included  to  provide  the 
engineer  with  the  knowledge  and  experience  of  experts  without 
making  their  constant  presence  mandatory. 

Once  a  manual  of  this  type  is  developed,  proper  intro¬ 
duction  into  the  project  becomes  a  major  challenge.  The 
natural  reaction  of  most  design  engineers  receiving  this  type 
of  formidable  document  is  to  file  it  for  future  reference. 

The  necessity  for  Indoctrination,  wherever  necessary,  becomes 
of  paramount  importance  at  this  point.  Once  development 
engineers  become  aware  of  the  usefulness  of  the  manual,  imple¬ 
mentation  becomes  self-sustaining.  However,  the  traditional 
inertia  against  adopting  new  thoughts  and  working  habits  has 
to  be  overcome. 

Existing  Problem  Areas 

In  an  objective  analysis  of  the  four  months  since  the 
introduction  of  our  new  manual,  several  problem  areas  become 
apparent.  Additional  clarification  has  been  found  to  be 
necessary  in  establishing  exactly  when  a  formal  meeting  should 
be  called.  Each  design  has  an  optimum  period  for  review, 
depending  on  such  things  as  complexity,  functional  require¬ 
ments,  and  caliber  of  cognizant  development  engineers.  Proper 
scheduling  will  result  in  maximum  economies  and  technical  con¬ 
tributions.  On  the  other  hand,  incorrect  scheduling  will  cause 
a  general  degradation  of  the  entire  design  review  effort. 

Another  area  which  has  warranted  careful  scrutiny  has 
been  the  manual  Itself.  At  present,  the  'itckllsts  are  pre¬ 
sented  in  a  manner  which  closely  follows  vhe  development 
cycle.  However,  the  need  for  cro33-ref erencing  related  iter 
of  various  checkll its  has  become  apparent.  Undoubtedly,  sr 
checklist  questior  can  be  rephrased,  expanded,  e  mbined, 
eliminated.  We  a  *  '  >w  in  the  prices'  of  re-evaluating  th- 
items.  In  addition,  it  is  f^it  tnat  subject  index  will  add 
considerably  to  the  usefulness  of  the  document. 

The  need  for  constant  Improvement  in  a  critical  activity 
such  as  design  review  cannot  be  overemphasized.  Seemingly 
minor  modifications  may  provide  the  extra  impetus  required  to 
develop  the  valuable  contribution  possible  from  this  endeavor. 


Summary  and  Conclusions 

In  order  to  Insure  adequate  design  reviews,  we  have 
developed  a  comprehensive  manual  which  contains  detailed 
checklists  and  design  methods.  The  primary  effort  has  been 
to  encourage  the  development  engineer  to  effectively  review 
his  design  a3  it  progresses.  At  various  milestones  in  the 
development  program,  formal  meetings  are  held  to  review  the 
adequacy  of  his  methods,  not  nece33arily  the  design  itself. 
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Design  review  conferences  have  grown  leas  time  consuming, 
with  analytical  computations  rather  than  intuitive  feelings 
forming  the  basis  for  discussions.  The  necessary  ingredients 
for  reliable  designs  have  been  more  clearly  defined,  removing 
a . f  the  previously  experienced  ambiguities.  However, 
further  areas  of  improvement  for  policies,  procedures,  check¬ 
lists,  and  design  methods  are  now  being  planned  for  the  in¬ 
creasingly  complex  systems  and  severe  development  schedules 
anticipated  in  the  near  future. 
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APPENDIX 


Item  1.  1  -.Design  Review  Phasing  Diagram 
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Item  2  -  Logic  Review  -  Electrical 


a.  Have  the  restrictions  Imposed  upon  perfor¬ 
mance  of  equipment  by  limitation  of  \ehlcles  carrying  the 
equipment  been  fully  evaluated?  Calculations  will  be  dis¬ 
cussed  at  the  design  review  meeting,  i.e., 

1.  RF1  due  to  functional  parts  of  vehicles 

2,  Flutter  and  vibration  natural  frequencies  and 
responses  of  the  vehicles  at  equipment  mounting 
points  In  each  axis  of  excitation. 

b.  Have  a  philosophy  and  a  system  of  Intercon¬ 
necting  grounds  been  established  to  avoid  undesired  inter¬ 
ference  effects?  Has  a  report  been  generated  defining  this 
philosophy  and  setting  up  the  ground  rules  for  Inter¬ 
connecting  grounds?  (See  Appendix  Item  1  of  Design  Review 
Manual,  ) 


c.  If  a  requirement  of  a  specification  is  not  con¬ 
sistent  with  the  existing  state  of  the  art,  are  all  risk 
areas  being  identified  and  personnel  being  assigned  to 

keep  track  of  progress? 

d.  Has  a  report  been  prepared  using  Value 
Engineering  techniques  to  determine  the  relative  actual 
value  of  each  function  specified  to  balance  the  cost  of 
each  function  specified  in  money,  weight,  schedule,  volume, 
reliability,  primary  power,  cooling  air,  and  maintainability, 
(See  Appendix  Item  6,  Design  Review  Manual.) 

e.  (Reference  Check  Ll3t  Item  No,  52,  8.1  System 
Logic  Review,  .Design  Review  Manual).  Has.  a  report  been 
prepared  which: 

1.  Lists  and  defines  all  functions? 

2.  Assigns  weighting  factor  to  each  function? 

3.  Establishes  reliability  of  each  Individual 
function? 

4.  Determines  over-all  reliability  from  formula? 
(Reference  Appendix  Item  10  of  Design  Review  Manual), 
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a.  Has  a  study  been  made  with  formal  analytical  compu¬ 
tations  to  determine  the  relative  merits  and  tradeoffs  of 
alternate  module  structural  concepts? 

b.  Where  vibration  schedule  inputs  in  vibration  exceed 
plus  or  minus  5  g  at  any  forcing  frequency,  has  an  effort  been 
made  to  determine  the  true  actual  dynamic  environment  at 
takeoff  by  tests  or  analysis? 

c.  Has  a  report  been  .generated  defining: 

1.  Categories  of- Units? 

2.  Types  and  mounts  of  cooling  available  for 

each  unit? 

3.  Ambient  surrounding  each  unit: 

Temperature  and  altitude? 

Thermal  shock? 

Pressure  variation? 

Pressure  variation  with  time? 

4.  Cooling  air: 

Temperature  versus  altitude? 

Maximum  change  of  temperature  with  time? 

Change  in  pressure  across  unit  and  tolerances? 

Change  in  pressure  versus  altitude  for  all 
governing  flight  conditions? 

Contaminant  content? 

d.  Have  computations  been  performed  determining  heat 
transfer  effectiveness  of  typical  units  and  modules,  as  per 
Appendix  Item  21  of  the  Design  Review  Manual  or  Item  8A  of 
this  paper? 

e.  Will  air-cooled  units  withstand  3  psi  differential 
pressure  with  plus  0.50  margin  of  safety  versus  permanent 
deformation  of  dust  cover  or  structural  members,  utilizing 
large  deflection  membrane  theory  of  hoop  stress  plus  bending 
stresses  a3  per  reference  R.  J.  Roark,  Second  Edition, 
"Formulas  for  Stress  and  Strain"? 

Item  4  -  Functional  Circuit  Review 

a.  Have  all  Input  and  output  signal  characteristics  been 
fully  coordinated  with  those  of  associated  blocks?  (See 
Appendix  Item  38  of  Design  Review  Manual  or  Item  8B  of  this 
paper.)  13  a  summary  of  such  signal  requirements  available? 
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b.  Typical  Circuit  Check  Llata 

Video  Circuit 

1.  Is  the  function  economically  achieved? 

2.  Ia  the  design  correct  so  that  the  ciroult 
will  do  what  Is  desired? 

3.  Will  it  function  correctly  over  the  range 
of  environment  and  component  tolerances? 

4.  Are  capacitors  within  voltage  ratings? 

5.  Are  resistors  within  power  ratings? 

6.  Are  relay  contacts  within  current  and 
voltage  ratings? 

7.  Are  diodes  within  forward-current  and 
back-voltage  ratings? 

8.  Are  transistors  within  current  and  voltage 
ratings  and  properly  bias-stabilized? 

9.  Are  amplifier  inputs  -and  "outputs  suf¬ 
ficiently  isolated  (physical  spacing; — — ^ 
common  leads,  power  supply  returns,  etc.) 
to  prevent  parasitic  oscillations? 

10.  Are  stray  capacitances  low  enough  to  avoid 
undeslred  time  delays? 

c.  Typical  Flip-Flop  Circuit  Check  List 

1.  Is  it  stable? 

2.  Does  it  respond  to  desired  flipping 
signal? 

3.  Does  it  abstain  from  responding  to  un¬ 
deslred  noise  spikes,  power  supply 
transients,  etc.? 

4.  Is  its  output  properly  (correct  voltage 
and  impedance)  coupled  to  its  load? 

5.  Are  its  parts  all  within  their  ratings? 
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6.  Are  appropriate  re-set  signals  connected  to  it? 

7.  Is  speed  of  response  proper  (neither  too  fast 
nor  too  slow)  for  desired  functions? 

d.  Have  pulse  widths,  circuit  delays,  and  pulse  shape 
deterioration  been  examined,  as  per  Appendix  Item  43  of  the 
Design  Review  Manual  or  Item  8c  of  this  paper,  to  establish 
that  signals  are  properly  synchronized? 

Item  5  -  Part  and  Module  Reviews 

a.  Is  natural  frequency  of  module  structure  (as  defined 
ir.  Appendix  Item  49  of  the  Design  Review  Manual  or  Item  8d  of 
this  paper)  greater  than  100  cps? 

b.  By  moment  distribution,  are  cross-sectional  module 
bending  stress  margins  of  safety  greater  than  plus  0.50,  a3 
defined  in  Appendix  Item  51  of  the  Design  Review  Manual? 

c.  Is  there  sufficient  coolant  to  reduce  the  temperature 
of  sensitive  members  to  their  safe  operating  temperature  with 
a  reasonable  margin  of  safety,  as  defined  in  Appendix  Items 

21  and  22  of  the  Design  Review  Manual? 

d.  Is  data  available  defining  derating  program  to  im¬ 
prove  estimated  component  reliability  numbers  due  to  tempera¬ 
ture,  electrical  stress,  vibration,  environmental  stress, 
aging,  humidity,  etc.?  This  data  will  be  presented  at  the 

design  review  meeting.  Hie  presentation  should  set  up  general - - 

ground  rules. 

e.  Are  transistors  or  subminiature  tubes  available  with 

1.  Frequency  characteristics,  as  per  specification? 

2.  Noise  figure  in  first  transistor  or  tube,  as 

per  requirement? 

5.  Requisite  stability  characteristics  with 
aging  and  temperature? 

4.  Sensitivity,  as  per  specification? 


/ 
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f.  How  does  module  perform  when 

1.  B  plus  is  varied  to  plus  or  mlnu3  10  per  cent? 

2.  Filament  voltage  (if  any)  is  varied  plus  or 
minus  10  per  cent? 

3.  B  minus  (if  any)  is  varied  'plus  or  minus 
10  per  cent?  (Reference  Appendix  Item  69, 
Design  Review  Manual) 


Item  6  -  Line  Replaceable  Unit  Review 


a,  _ Has  a  report  been  generated  summarizing  all 
unit  hotspots,  all  heat  generators  in  unit,  and  all  critical 
heat-sensitive  parts?  Is  the  report  adequately  quantitative 
so  as  to  include  location  of  critical  parts,  time-rate 
generation  of  heat,  degradation  of  performance  with  tempera¬ 
ture  and  time,  etc.? 


b.  Has  analysis  been  made  to  solve  for  heat 
transfer  coefficient  (integrated  for  entire  unit),  as  per 
Appendix  Item  77,  Design  Review  Manual? 


c.  Has  a  test  and  analysis  report  been  generated 
attempting  to  define  the  true  dynamic  environment  direction 
for  various  suitable  aircraft  with  respect  to: 


1.  Location  of  equipment  in  airplane? 


ferentlnl 


-axo_1  tat  ion? 


3.  Cumulative  cycles  at  each  frequency? 
Bandwidth  frequency  spectrum? 

5.  Acceleration  input? 

d.  Will  air-cooled  unit  withsand  3  psi  dif- 

pressure  without 

1.  Excessive  leakage  of  coolant  through  gaskets, 
etc  .  ? 

2.  Development  of  gap3  which  may  permit  RFI 
leakage  ? 

3.  Permanent  deformation  of  dust  cover  through 
membrane  diaphragm  action? 
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Item  7  -  System  Worth  Review 

a.  Have  all  Input  and  output  3ignal  characteris¬ 
tics,  as  defined  In  Appendix  Item  80,  Design  Review  Manual, 

fully  coordinated  with  those  of  associated  units? 
Summary  of  such  signal  requirr.nents ,  as  per  Appendix,  will 
be  furnished  at  design  review  meeting. 

b.  Is  system  output  high  enough  to  guarantee 
producing  desired  effect?  Is  noise  level  of  system  output 
low  enough  not  to  produce  a  false  alarm  or  other  undesired 
effect?  Reference  Appendix  Item  80,  Design  Review  Manual. 

c.  Have  all  the  power  leads  been  determined  to 
be  located  at  the  maximum  distance  from  the  signal  leads 
that  the  physical  configuration  of  the  packaging  will  allow? 

d.  Has  the  packaging  technique  provided  for 
flexibility  in  design  which  will  enable  the  system  to  grow 

in  complexity  or  be  reduced,  at  the  request  of  the  customer, 
without  affecting  the  units  that  would  not  be  changed? 


Item  8  -  Design  Methods 

A.  Effectiveness  of  heat  transfer  = 
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In  test,  monitor  surface  temperatures  of 
critical  components. 

Onerate  modules  and  units. 

Completely  insulate  module  or  unit  to  minimize 
conductive  and  radiation  heat  losses. 

Unit  generates  own  heat. 

2 

Supply  cooling  air  until  maximum  temperature  of 
critical  air  is  obtained. 


Compute  effectiveness 


1 .  Impedance 

Impedance  of  Interconnections  should  be 
low  to  minimize  capacitive  pickup  from  one  lead  to  another. 


a.  Circuit  impedance  should  be  as  low 
as  possible  to  minimize  capacitive 
pickup  of  interference,  degradation 
of  pulse  shapes,  undesired  feedback. 
Where  pos3 ible, impedances  should  be 
held  to  between  1000  and  10,000  ohms, 
maximum. 


b.  Stray  capacitances  should  be  mini¬ 
mized  by  locating  components  to  keep 
critical  leads  short.  Try  where 
possible  to  hold  leads  to  1  inch, 
maximum  length. 

2 .  Amplitude 


a.  Is  circuit  design  capable  of  handling 
existing  amplitudes  without  over¬ 
loading,  instability,  and  waste  of 
power?  e.g.,  Amplifier  (transistor) 

B  +  —  i.5  (a)  v1 

where  B  =  Collector  supply  voltage 

G  -  Voltage  gain  of  circuit 

Vj  •  Input  voltage 


'la  the  output  background  noise  level  tolerable  to  the 
input  to  which  it  is  connected0 
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'■>.  In  output  .signal  of  one  circuit 

reliably  sufficient  to  perform  de¬ 
sired  function  an  _J.nput  to  another 
circuit? 


c.  Ia  circuit  uneconomical  in  using 

excessive  amplitude  of  signals  con¬ 
necting  one  box  to  another? 

3 .  Pul3e  Duration 

Pulse  duration  which  13  trying  to  trigger 
d  flip-flop  or  produce  other  action  should  be  long  enough 
to  produce  desired  result  with  adequate  margin  of  safety 
(e.g. ,  +1.50),  taking  into  account  all  tolerances  affecting 
desired  result  —  resistors,  effect  of  stray  capacitance, 
tolerance  on  transistor  parameters,  temperature  effects,  etc. 

;4 .  N0I3  e 

Compute  noise  level  of  input  signal,  e.g,. 


Thermal  noise  voltage  at  input 


N  =  Ci  x  Gr 


where  N  -  Noise  at  input  to  circuit 
Ci  a  Noise  figure  of  receiver 
Gr  »  Gain  of  receiver 


N  •  Vi  (Z^~ZC) 


where  N  =  Undesired  induced  noise  voltage 
■  Voltage  level  of  nearby  inter¬ 
fering  circuit 
Zc  -  Impedance  of  undesired 
capacitive  coupling 
Z  «  Impedance  to  ground  of  circuit 
under  investigation 
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Inductive  pickup  noise  voltage 

N  *  Inductive  pickup  noise  voltage 
-  Current  in  nearby  wire 
M  *  Mutual  Inductance  between  wire 

carrying  undesired  signal  and  wire 
under  investigation. 

2 i  =  Input  impedance  of  circuit  to  which 
signal  is  being  fed. 

Z0  ■  Output  impedance  of  circuit  from 
which  signal  i3  being  fed. 

Z, 

H  -  M(l4)  1 

^i  ^o- 

Interference  noise  produced  by  common 
power  supply  and  ground  return  Impedances  between  module 
under  consideration  and  interfering  modules: 

Let  N  -  Interfering  voltage  actual 

actual 

Zj_  -  Common  Impedance 

Ij^  -  Undesired  current 

K  -  Derivative  of  circuit  output 

voltage  with  respect  to' 
induced  undesired  voltage 

^actual  s  ^izc 
^actual  <C  ~  Et 

Where  Lt  -.Threshold  signal  that  will 

operate  flip-flop  or  produce 
-  other  undesired  effects  of 
tolerances  and  environment 

Step  i/1  -  Determine  noise 

Step  i/2  -  Determine  square  root  of  sum  of  squares 
of  noise  voltages. 

Step  -'3  -  Compare  result  of  Step  #2  with  amplitude 
of  signal  appearing  at  that  point  in 
the  circuit  under  investigation. 

Step  //'i  -  Determine  margin  of  safety  versus  false 
alarm."  and  other  undesirable  effects 
exceeding  tolerable  limits. 
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5.  Circuit  Delays  and  Pulse  Shape 

Seterlo'ratlon 

It  Is  recommended  that  individual  circuit 
deJav*  not  exceed  5  per  cent  of  the  mean  pulse  width 
employed  in  the  computer. 

Delays  are  additive:  rise  time  contri¬ 
butions  combine  approximately  as  the  cube  root  of  the  sum 
of  the  cubes. 


Tr  -  Rise  time  of  output  of  first  of  a  chain 
nl  of  circuits  when  input  has  2ero  rise 
time . 

Tr  -  Rise  time  of  output  of  chain  of  circuits. 
Pp  =  Rise  time  of  input. 


It  is  recommended  that  teat  measurements 
of  the  following  be  made  on  oscilloscope  input  and  output 
signals  of  adjacent  major  circuits  : 

1.  Pulse  Shape:  Rise  and  fall  time  of 

signal . 

2.  Pulse  Width:  Time  between  half  amplitude 

points . 

3.  Circuit  Delay:  Time  between  half  ampli¬ 

tude  at  input  and  half 
amplitude  at  output.  Then, 
it  is  necessary  to  compare 
the  output  signal  of  the 
drive  circuit  with  the 
desired  input  to  the  driven 
c ircuit . 

C.  n  pulse  must  occur  during  a  gate  signal. 

Does  gate  precede  and  succeed  beginning  and  end  of  pulse  by 
at  least  2‘j  per  cent  of  pulse  v.-idth?  Have  circuit  delay 
tolerances  beer  analysed  similar  to  oulsc  a—nlitude 
analysis  as  per  quest  1  on  wo.  t ,  Check  List  8.-.  of  Design 
Review  Manual? 
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Synchronizing  (information  retiming) 

The  Inherent  time-delay  properties  of  most  com¬ 
puter  components,  and  the  unequal  path  length  of  logic 
oi.cu.'try  that  various  information  must  pass  through,  cause 
different  delays  In  arrival  of  two  signals  to  a  loglo 
point.  Therefore,  measures  must  be  taken  to  assure  syn¬ 
chronization  of  various  operations  of  the  computer. 

The  use  of  a  highly  stable  crystal-oontrolled 
olock  pulse  generator  which  also  serves  as  a  time  reference 
may  help  to  accomplish  synchronization  of  the  pulses. 

D.  It  is  requisite  to  determine  the  natural 
frequency  of  each  significant  type  module.  The  defleotiuns 
and  flexibilities  of  the  modules  are  functions  of  the 
natural  frequencies,  whioh,  in  turn,  are  functions  of  the 
distributions  of  masses  and  inertias  of  the  modules. 

y  ■  a  aln  wn  t 

where  y  ■  Amplitude  of  module  (inch). 

a  »  Maximum  single  amplitude  (inch). 
wn  -  Natural  frequency  (radians/second ). 
t  -  Time  (second). 

y  -  (- y)wn2 

Determine  module  natural  frequency. 

Step  1  -  Determine  all  dimensions  of  the  module.  What  are 
”  the  boundary  conditions,  simply  supported  or 

fixed?  Locate  all  masses  in  the  module.  Set  up 
reasonable  mathematical  models  and  load  paths. 

Step  2  -  Compute  the  moment  of  inertia  of  a  typical  cross- 
section  of  the  module  about  a  principal  axis  of 
the  cross-section. 

Step  3  -  At  first  load  path,  consider  the  module  as  a  box 
beam  and  compute  the  static  deflections  at  every 
mass  point  due  to  1  p  leads.  Include  both  flexual 
and  shear  deflections. 

Step  ^  -  Compute  first  approximation  of  the  natural 

frequency  of  the  module  by  the  formula  shown  in 
Appendix,  Physical  Design  Review,  Design  Review 
Manual. 
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RELIABILITY  EDUCATION* 

by 

Harry  G.  Romlg 
Reliability  Director 
Military  Products  Division 
Hoffman  Electronics  Corporation 


ABSTRACT 


This  paper  covers  the  fundamentals  of  Reliability  Education.  It 
contrasts  the  desires  and  contributions  of  Industry  with  the  educa¬ 
tional  developments  and  offerings  of  the  Universities.  The  need  for 
Human  Engineering  is  pointed  out  and  also  for  simpler  and  improved 
evaluation  techniques  for  Reliability.  When  should  action  be  taken 
wi«-h  respect  to  components  or  the  system  as  a  whole  due  to  lack  of 
control  with  respect  to  its  reliability?  Supplementary  educational 
material  is  presented  indicating  different  methods  for  establishing 
statistically  sound  control  limits  based  on  desired  Confidence  Levels. 
An  educational  and  training  program  for  management,  engineering, 
purchasing,  production  and  other  associated  groups  is  outlined. 

Emphasis  on  the  team  approach  used  in  Operations  Research  and 
in  many  current  contract  negotiations  is  given,  together  with  con¬ 
sideration  of  the  contributiona  that  are  being  made  by  committees 
and  groups  working  in  logistics,  maintenance,  specifications,  testing 
and  research.  Economic  standards  of  quality  are  discussed  and  in 
addition  the  methods  to  be  used  to  educate  Individuals  in  this  field 
so  that  close  liaison  may  be  maintained  between  standards,  specifi¬ 
cations,  design,  value  engineering,  fabrication  and  reliability  in  all 
its  phases.  Finally  the  educational  precepts  and  methods  for 
scientific  and  reliability  education  are  noted  and  methods  of  imple¬ 
menting  these  are  proposed. 

1.  The  Reliability  Problem 

The  need  for  reliability  in  its  different  aspects  has  been 
apparent  for  centuries.  Protection  was  needed  -In  early  times  against 
nature  and  the  wild  beasts.  Homes  were  built  in  caves  and  elsewhere 
and  protective  weapons  were  made.  Clubs  and  spears  were  produced  that 
were  deemed  inherently  reliable.  Later  bows  and  arrows  were  developed 
that  would  function  without  too  many  failures  in  hunting  food  or  in 
battles  with  enemies  of  all  kinds.  Men  and  allies  were  selected  as 
team  mates  or  workers  bee 'use  of  their  Inherent  integrity.  They  also 
were  considered  reliable  if  they  proved  to  be  effective  allies.  Eventa 


♦Paper  presented  at  the  Tuesday,  February  16,  1960  morning  session  of 
the  6th  Joint  Military-Industry  Guided  Missile  Reliability  Symposium, 
February  15-17,  1960  at  Fort  nliss,  El  Paso,  Texas. 
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are  expected  to  occur  according  to  some  fairly  well  defined  pattern. 

Some  deviations  are  expected.  If  one  unit  or-event  1b  widely 
different,  sometimes  termed  the  maverick  of  the  group,  the  future 
performance  of  other  units  in  the  group  is  questioned.  What  steps 
c. ..  L  .  taken  to  reduce  the  probability  of  the_occurrence  of  the 
unusual?  Also,  what  information  is  required  to  determine  how  wide  a 
band  of  unreliability  exists?  Can  Individuals  be  trained  to  dlffer- 
antiate  between  cause  systems  that  produce  satisfactory  events  or 
products  and  those  cause  systems  that  contain  unfortunate  tendencies 
towards  the  production  of  erratic  events  and  performances?  Being 
able  to  control  the  environment  in  which  we  live  is  extremely  impor¬ 
tant.  Being  masters  of  the  situations  that  arise  rather  than  being 
controlled  by  the  current  events  or  by  irresponsible  Individuals 
requires  knowledge  and  training.  Reliability  means  stability, 
dependability  and  adequate  controls.  Thus  education  that  develops  an 
appreciation  of  how  to  attain  this  valuable  feature,  namely  relia¬ 
bility,  will  contribute  greatly  to  the  national  welfare. 

The  generalizations  above  point  out  the  necessity  for  apprecia¬ 
ting  what  the  reliability  problem  is  and  what  steps  may  be  taken  to 
achieve  the  reliability  desired.  Events,  products,  actions  are 
dependent  upon  many  causes.  One  group  deals  with  another  group.  If 
those  dealings  are  mutually  beneficial,  these  relations  are  continued. 
If  practices  are  sharp,  products  bought  are  unsatisfactory,  schedules 
are  delayed  unduly,  and  the  liaison  results  in  adverse  reactions, 
then  relations  are  severed.  ignorance  may  be  the  cause.  Lines  of 
communication  have  broken  down,  no  agreements  hava  bean  rsached  that 
axe  so  staple  — *  a*.  pciitbLe  ffereaoe*  i.a  reactises  mmj  not.  oecer  - 
thus  lack  of  confidence  and  a  feeling  of  uncertainty  exists.  Products 
made  or  shipped  tend  to  be  of  such  poor  quality  that  they  would  ordin¬ 
arily  be  returned  except  that  they  are  needed  to  meet  schedules.  The 
result  la  an  unreliable  assembly  or  system.  The  meaning  of  reliability 
dependability,  service,  longevity  is  not  clear.  Hence,  some  educa¬ 
tional  program  is  required  to  develop  a  better  knowledge  of  reliability 
and  its  meaning,  and  also  to  introduce  effective  measures  to  secure  a 
reliable  product. 

The  reliability  problem  is  as  old  as  civilization.  Now  it  is 
being  stressed  since  allowable  variations  are  being  narrowed.  parts 
and  performance  must  be  satisfactory.  In  each  field  custom  has 
developed  a  standard  of  some  kind  which  is  considered  economic.  If 
achieved  then  the  notion  of  reliability  in  that  area  has  certain 
definite  connotations.  If  not  achieved  the  causes  of  failure  must  be 
determined.  Weighting  these  causes,  taking  proper  steps  to  reduce  the 
failures  or  potential  failures  to  a  minimum  -  all  are  functions  that 
must  be  understood  Training  is  needed.  What  type  of  education  is 
beet  suited  for  this  task?  Consideration  will  be  given  to  both 
desirable  and  possible  undesirable  elements  in  Reliability  Education 
as  it  exists  today.  This  lays  the  groundwork  for  a  strong  rudimen¬ 
tary  training  program  in  Reliability. 


.  Elements  in  Reliability  Education 


Quality  Control  as  applied  in  industry  was  started  by 
Dr.  Walter  A.  Shewhart  of  the  Bell  Telephone  Laboratories  in  1924 
and  was  extended  into  commercial  practices  by  Messrs.  Harold  P.  Dodge, 
George  D.  Edwards,  Donald  A.  Quarles,  Dr.  Paul  M.  Olmstead  and  many 
others  including  the  author  in  the  Bell  System  and  in  a  few  other 
industries.  It  was  taken  ever  into  Military  operations  about  1941 
and  in  the  Aircraft  and  Missile  Industries  was  part  of  many  contracts 
*hich  included  MlL-Q-5923  (original,  A,B  &  c  issues)  as  mandatory. 

It  is  now  replaced  by  MIL-Q-9858  issued  by  the  Department  of  Defense. 
Many  in  Research  and  Development  work  do  not  feel  that  MIL-Q-9858 
covers  R  &  D  work  as  well  aa  Phase  A  of  MIL-Q-5923C.  The  Quality 
Control  specification  did  not  result  in  the  Reliability,  Dependability 
and  Longevity  including  ability  to  perform  after  5  years  shelf  life, 
that  was  desired.  Maintainability  also  was  not  deemed  to  be  covered 
as  well  as  it  should  be.  Reports  were  issued  covering  certain  areas 
indicating  replacement  costs  were  often  as  high  as  10  times  the 
original  costs.  Action  was  warranted.  Reliability  was  selected  as 
the  factor  that  must  be  stressed. 

Some  concerns  already  had  decided  that  Quality  Control  was  not 
sufficient.  However,  many  did  not  really  know  what  Quality  Control 
should  do,  what  its  functions  actually  were.  _ Many  renamed  their 
Inspection  Departments  Quality  Control  and  wrote  a  manual.  Some 
Included  Test  in  inspection,  renamed  Quality  Control,  but  few  had  any 
Quality  Control  functions  in  Engineering,  or ^Research  and  Development. 
Others  needed  to  use  smaller  samples  so  took  their  clue  from  Research 
and  Development  customs  and  used  small  samples  or  no  samples  for 
intermediate  tests.  Others  acquainted  with  more  recent  techniques 
added  "Statistical  Quality  Control"  mentioned  in  MIL-Q-5923.  Section 
4  of  the  Military  Specifications  was  titled  "Quality  Assurance"  so 
Quality  Assurance  departments  were  organized.  Their  functions 
differed  considerably  from  those  covered  by  the  Quality  Assurance 
Department  of  the  Bell  Telephone  Laboratories. 

Next  came  a  series  of  Reliability  specifications  covering  differ¬ 
ent  areas.  Some  were  preceded  by  the  "AGREE"  (Advisory  Group  on 
Reliability  of  Electronic  Equipment)  Report  covering  Reliability 
requirements.  A  series  of  Military  specifications  and  Exhibits  cover¬ 
ing  Reliability,  Maintainability  and  allied  areas  were  and  are  being 
issued,  some  are  now  incorporated  into  Missile  end  Electronic 
Equipment  contracts.  Many  conferences  and  symposia  on  Reliability  have 
been  held,  this  being  one  of  a  series. 

The  above  background  indicates  roughly  the  historical  background 
of  Reliability.  What  elements "should  be  included  in  a  Reliability 
Educational  program?  Which  are  most  critical?  These  should  be 
tabulated  and  analysed  so  that  training  effort  may  be  applied  most 
efficiently . 
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The  following  elements  should  be  considered: 
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Definitions  and  description  of  Reliability. 

Mathematical  Models  for  Reliability  Evaluations. 

Reliability  of  Materials  -  testa,  checXs  and  controls. 

Reliability  of  Components  -  teats  and  measurements, 
longevity. 

Mean  Time  Before  Failure  (MTBF)  -  meaning  and  use. 

Minimum  Time  Before  Failure  (mtbf)  -  distribution  theories 
for  minimum  and  maximum  values. 

Confidence  Limits  -  Degree  of  Belief. 

Mission  Times  -  Nature  of  Missions:  Three  possible 
categories: i 

(1)  Instant  Response  -  immediate  reaction  and  performance. 

(2)  Short  Time  performance  -  dependable  response. 

(3)  Long  Time  performance  -  requirements  for  space 

operations. 

Variables  and  Attributes  data  -  Evaluation  methods. 

Analysis  of  variance  -  Experimental  Models  -  Securing 
Reliability  data  by  planned  experiments. 

Reliability  in  Design  and  Engineering. 

Reliability  in  Pilot  Runs. 

Reliability  in  Production. 

Reliability  in  purchased  parts. 

Reliability  Tests  and  Environmental  Tests. 

Bench  Tests  and  Flight  Tests  -  Reliability  evaluations. 

Malfunction  Reports  -  Use  and  Benefits. 

Field  Reports  and  Unsatisfactory  Reports  (UR's). 

Standards  -  Economic  Levels;  Choice  of  Parts;  Suppliers; 

Sources. 

In  Military  contracts  -  Handling  of  Nonstandard  Parts. 

Bills  of  Materials  -  Lists  of  Standard  and  Nonstandard 
Parts. 

Accelerated  Tests  -  Life,  Longevity  and  Shelf  Life. 

Data  Processing  for  Reliability. 

Reliability  and  its  Satellites  -  Quality  Control,  Quality 
Assurance,  inspection.  Test,  Testing  Equipment,  Calibra¬ 
tion,  primary  and  secondary  Standards,  Standards  of 
Performance. 

Applications  of  Designs  of  Experiments  and  Operations 
Research  - 

Present  Reliability  Organizations. 

Changes  in  Design  and  Specifications  -  Part  played  by 
Engineering,  Reliability  and  Design  Review  Boards. 

Need  for  unbiased  reports  on  Reliability  -  Independent 
versus  non-independent  organizations. 

Project  System  of  Reliability  -  faults  and  merits. 

Centralized  system  of  Reliability  -  faults  and  merits. 

Independent  Laboratory  System  for  Reliability  Evaluations  - 
Benefits  and  losses. 
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32.  Ideal  Quality  Engineering  Organisation. 

33.  Manuals  and  instruction  Books  in  Reliability. 

34.  Committees  and  Conferences  for  Reliability. 

35.  Training  for  Reliability. 

-laments  in  Logistics  and  Operations  Research  may  be  added  as 
they  contribute  heavily  in  many  areas.  The  detailed  elements  above 
can  be  covered  by  extensive  training  programs.  Human  engineering  la 
written  in  but  its  effect  is  difficult  to  evaluate.  Training  in 
btaining  data  before  and  after  human  engineering  elements  or  other 
parts  have  been  added  is  required.  For  example,  some  have  reported 
that  on  certain  vacuum  tubes,  removing  the  old  JAN  shields  improves 
the  Reliability  10  to  20%  roughly,  but  adding  the  new  MIL  shields 
increases  MTBF  values  from  1,000  hours  to  6,000  hours.  Costs  are 
increased  for  the  initial  installation  but  overall  coats  are  decreased 
when  maintenance  coats  are  included. 

-  Industry  Looks  at  Reliability  -  Contract  Requirements 

Industry  seeks  a  profit.  Our  economic  system  is  geared  to 
extensive  growth  patterns  with  population  expanding,  number  and 
varieties  of  products  being  continually  augmented,  national  wealth 
Increasing,  defensive  and  offensive  military  needs  continually  chang¬ 
ing  in  line  with  scientific  improvements.  Looking  over  contracts  for 
military  products  Including  missiles,  why  should  there  be  any  relia¬ 
bility  paragraphs  added?  The  products  to  be  delivered  are  supposed 
to  function  and  perform  the  tasks  noted  in  the  contracts.  If  all 
features  of  these  contracts  were  met.  there  would  apparently  be  no 
need  for  reliability  ri juirements. 

•  If  the  designer  has  made  the  best  possible  economic  unit  within 
the  present  state  of  the  art,  the  tolerances  allowed  are  reasonable, 
and  the  beat  possible  reliable  and  quality  parte  are  called  out  in 
the  bill  of  materials  and  have  been  secured  from  the  most  reliable 
vendors  in  the  field,  a  reliable  and  quality  product  should  be 
obtained  and  require  practically  no  checking.  A  satisfactory  product 
is  assured  since,  with  all  parts  at  minimum  or  maximum  values  in  an 
unfavorable  direction  one  way  and  at  maximum  and  minimum  values  in  an 
unfavorable  direction  the  opposite  way,  all  such  assembled  units  have 
been  found  by  tests  to  operate  satisfactorily.  In  addition,  other 
tests  show  they  have  no  current  failures  and  after  25  to  75  months  of 
shelf  life  still  operate  if  placed  in  service.  Also,  no  failures  in 
operations  occur  for  25  to  100  months  for  all  units  accepted,  with 
such  a  history  then  no  Inspections  and  tests  other  than  routine 
Inspections  or.  the  line  may  be  required  during  fabrication.  Such  may 
be  the  case  for  an  automatic  production  line  wherein,  if  necessary, 
automatic  measurements  may  be  made  at  certain  stages  of  the  assembly 
and  all  nonconforming  parts  are  rejected  by  the  machine  automatically. 
This  is  the  Ideal  setup  for  continuous  production  and  may  be  very 
satisfactory  for  the  majority  of  simple  component  parts  such  as 
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r«pacltrrsi  resistors,  diodes  end  the  like.  Some  commercial 
r i sistora  have  been  made  for  years  by  such  machines.  Some  shoes 
also  are  practically  made  automatically  by  machines  obtained  by 
rental  from  a  general  corporation  specialising  in  automatic  mach- 
?— r**  Such  is  true  for  the  production  of  bottles,  one  production 
line  even  made  cheap  alarm  clocks  automatically.  For  electronic 
equipment  some  parts  such  as  potentiometers,  diodes  and  even  some 
modules  may  be  made_by  unit  construction  methods.  When  automatic 
assembly  and  fabrication  era  used,  units  are  produced  so  rapidly 
that  automatic  inspection  devices  must  be  installed  at  strategic 
points,  samples  of  checked  parts  must  be  made  to  provide  assurance 
that  the  automatic  testing  devices  maintain  their  accuracy  and  that 
they  are  calibrated  properly  and  at  appropriate  intervals  against 
primary  standards.  Also  their  precision  must  be  good,  i.e. 
successive  variations  in  repeat  readings  must  be  maintained  smell 
with  allowable  maximum  variation  permitted  leas  than  10%  of  total 
tolerance  band  width  with  1%  preferred. 

With  the  above  facts  noted. industry  accepts  these  reliability 
requirements  and  adds  the  Reliability  Testa  as  necessary  costly 
tests  that  it  believes  might  be  eliminated  if  the  customer  would 
only  be  satisfied  with  the  assurance  that  performance  and  flight 
tests  will  eliminate  the  weaker  elements.  There  exists  one  group 
In  industry,  still  operating  in  spite  of  improvements  in  available 
techniques,  that  use  trial  and  error  methods.  Their  contention  is 
that  performance  is  the  key  check  for  a  satisfactory  product,  hence, 
make  a  series  of  different  types  of  units,  operate  them,  then  select 
the  one  that  operates  best.  Analyses  of  test  results  for  various 
characteristics  and  environments  show  that  such  units  work.  Ship 
these  to  customers  but  assume  no  responsibility  for  future  operations. 
If  failures  occur,  rework  and,  if  necessary,  redesign.  The  argument 
is  that  the  changes  in  the  art  are  so  rapid  for  electronic  equipment 
that  units  become  obsolescent  more  rapidly  than  they  even  approach 
half-life  and  few  ever  reach  the  point  where  shelf  life  is  an  impor¬ 
tant  factor.  In  some  areas  this  contention  is  valid  but  nevertheless 
some  assurance  must  be  provided  Bhowing  the  product  is  satisfactory. 

The  Educational  Program  for  Reliability  must  be  geared  to 
consider  these  adverse  reactions.  Typical  of  these  antagonistic 
feelings  against  outside  Inspections  and  tests  are  the  following. 

Each  instrument  payload  for  scientific  investigations  in  apace  Is 
different  from  the  next  payload.  Each  engineer  and  designer  makes 
his  portion  of  the  whole  as  reliable  as  possible  within  the  area  of 
his  knowledge  of  operations.  The  scientist  or  engineer  working  on 
the  design  confers  with  his  colleagues  and  has  a  one  unit  assembly 
different  from  any  other  unit  and  one  that  probably  may  never  be  mads 
again.  He  firmly  believes  that  there  is  no  need  for  additional  tests, 
reliability  evaluations,  consideration  of  components,  tabulation  of 
expected  performance  for  parts,  hence,  additional  costs  incurred  by 
reliability  are  unnecessary.  Reliability,  Quality  Control  and  similar 
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techniques  only  apply  to  mass  production  of  parts  and  even  then 
only  in  a  limited  way.  Their  efforts  impede  the  schedule.  Design 
reviews,  even  by  recognized  experts  brought  In  as  consultants,  are 
a  waste  of  time  and  money.  Such  may  save  thousands  of  dollars  by 
recommending  early  changes  in  designs  that  will  never  work  or  will 

inefficiently.  Costly  rework  will  be  reduced  and  unit  costs 
may  be  reduced  through  Value  Engineering  so  that  the  company  making 
the  part  is  in  a  strong  competitive  position _when  contrasted  with 
similar  products  made  by  competitors. 

The  demands  of  Industry  are  for  simple  techniques  to  evaluate 
the  reliability  of  parts  and  assemblies  in  mass  production.  The 
models,  preprototypes  and  breadboard  designs  made  in  Research  and 
Development  undergo  such  critical  examinations  by  the  design  engineers 
and  technicians  when  being  built  that  auxiliary  reliability  checks 
ar«  considered  a  farce  in  many  cases.  A  goodReliability  Engineer 
vi'.l  realize  that  this  point  of  view  is  valid  in  a  great  many  cases. 
Making  tests  with  little  meaning  and  no  possible  constructive  reactions 
may  satisfy  your  customer  that  reliability  action  is  taking  place  when 
such  tests  add  nothing.  There  exists  a  need  for  constructive  training 
In  this  area. 

Reliability  organizations  provide  basic  services  to  design 
engineers.  Interchange  of  date-covering  all  parts  and  module  assemblies 
should  be  established  in  a  central  standard  agency  on  a  non-profit 
basis.  Establishing  such  a  central  source  for  pertinent  design  infor¬ 
mation  on  parts  and  components  of  all  kinds  provides  the  basic  elements 
for  a  good  design.  Limitations  on  certain  parts  must  be  recognized  and 
the  design  must  reflect  these  limitations  by  strengthening-  other 
supporting  dements.  The  Reliability  Education  program  should  cover 
this  service  and  provide  training  so  that  maximum  use  may  be  made  of 
such  data. 

Industry  appears  to  like  Panel  discussions.  Many  training 
programs  include  several  sessions  for  a  panel  discussion  of  selected 
topics.  If  handled  properly  by  a  good  moderator  it  is  very  educa¬ 
tional.  With  a  poor  moderator  many  panel  discussions  waste  the  time 
of  those  in  attendance.  Formal  training  with  scheduled  classes  is 
preferred  for  in-plant  or  out-of-plant  training  when  the  demand  for 
such  training  is  extremely  urgent.  Generally,  Industry  expects  the 
Universities  to  provide  formalized  training  in  Reliability  and  kindred 
fields  and  particularly  in  the  mathematics  and  statistics  required  for 
reliability  engineers.  Contract  stipulations  sometimes  require 
Industry  to  write  procedures,  instructions,  training  programs,  make 
moving  picture  films,  and  perform  other  tasks  required  to  implement  a 
large  complex  system.  These  are  often  considered  as  merely  instruc¬ 
tions  whereas  a  great  many  of  these  tasks  result  in  fully  oriented 
training  programs.  industry  absorbs  a  minimum  cf  these  tasks  and 
desires  that  the  Universities  assume  the  responsibility  of  covering 
the  remaining  areas  for  Reliability  Education. 
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The  Universities  are  continually  revising  their  curricula. 

More  courses  on  more  subjects  are  being  added.  In  the  Reliability 
=  I  '  the  university  contributions  to  the  program  consist  of  regular 
courses  in  Engineering  Statistics.  Mathematical  Statistics,  Quality 
Control,  Operations  Research,  Point  Sets,  Industrial  Engineering, 
Management  Courses,  and  special  courses,  such  as  Logistics,  Analysis 
of  Variance  and  Decision  Functions  given  by  the  Economics,  Mathematics, 
Engineering,  Physics  or  even  Business  Administration  Departments. 

Many  evening  courses  are  given  through  extension  although  some 
universities  give  accredited  night  courses  in  special  areas  such  as 
guided  missile  design,  propulsion,  instrumentation  and  industrial 
processes,  using  as  lecturers  industrial  experts,  many  of  whom  only 
give  one  course  each  year  or  each  semester.  Two  day,  one  week,  two 
week  or  several  weeks  of  special  conferences  are  provided.  For  the 
shorter  periods  special  groups  meet  together  for  two  days  to  two  or 
three  weeks  and  work  co-operatively  in  one  or  two  fields.  Some 
universities  even  cover  Reliability  in  somewhat  similar  concentrated 
Conferences  or  Training  Programs. 

The  Military  have  arranged  with  some  universities  to  provide 
complete  college  programs  in  these  areas,  such  as  reliability  and 
maintainability,  that  are  usually  miased  otherwise,  some  special 
conferences  are  arranged  through  special  trust  funds  such  as  a  recent 
conference  on  Logistics  at  Columbus,  Ohio.  Recently  in  Los  Angeles  a 
combination  University  and  Industry  Panel  covered  in  two  days  techni¬ 
ques  for  specification  writing.  Quality  Control  is  covered  by  a 
large  number  of  Universities  in  special  two-week  all-day  courses 
arranged  at  times  where  industry  may  be  able  to  send  tholr  men  there 
for  full-time  training  or  for  essentially  a  refresher  course.  Men 
from  industry  are  on  the  advisory  boards  of_the  Universities  and 
assist  them  in  establishing  short  courses  that  will  be  moat  beneficial 
to  the  local  industries. 

Technical  societies  are  holding  regular  periodic  evening  meetings, 
conferences,  conventions  and  panel  discussions.  The  various  university 
professors  that  are  conversant  with  statistics,  some  phases  of  relia¬ 
bility,  industrial  engineering  processes,  formal  and  applied  mathema¬ 
tics  and  various  kindred  fields  of  engineering  and  administration 
contribute  heavily  to  these  conferences,  etc.  Training  courses  have 
been  sponsored  by  many  technical  societies.  -University  professors 
conduct  many  of  these  courses.  They  also  act  as  consultants  on  many 
projects.  Unfortunately,  in  many  areas,  those  elements  that  contri¬ 
bute  to  quality  and  reliability  are  not  even  mentioned.  Such  elements 
are  taken  for  granted.  it  is  assumed  that  if  parts  are  made  right, 
and  designs  are  riqht,  then  these  parts  must  be  reliable. 

Reliability  Education  programs  should  be  arranged  at  periodic 
Intervals  In  the  universities  where  the  professors  meet  so  that  there 
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wi 1 L  be  more  consciousness  of  the  need  to  stress  reliability  in  all 
its  phases.  This  approach  might  increase  the  effectiveness  of  the 
universities  in  this  field.  Reliability,  being  taken  for  qranted, 
is  not  achieved  because  there  is  no  formal  effort  to  present  its 

in  the  various  university  programs,  where  quality-conscious¬ 
ness  exists,  reliable  products  result. 

5.  Reliability  Controls 

xn  Reliability  training,  emphasis  generally  is  placed  upon  the 
techniques  and  not  the  fundamentals.  Consequently,  when  one  specific 
part  or  component  fails  more  than  once  it  is  desired  to  know  whether 
such  a  part  failure  is  significant  or  not.  It  is  necessary  to  first 
develop  control  charts  for  Reliability  values  that  have  meaning  and 
to  establish  expected  levels,  expected  variances,  expected  standard 
deviations  and  to  choose  selected  Confidence  Levels.  This  selection 
is  not  as  easy  as  it  looks  since  there  are  a  large  number  of  variables 
involved.  Many  of  these  are  not  known  as  completely  as  is  desired. 
Consider  a  Reliability  value  of  0.99.  Using  the  theory  developed  by 
the  Task  3  Group  for  the  AGREE  report  involving  Mean-Time-Between- 
Fallures  (MTBF)  where  from  equation  (2)  of  page  177  of  the  AGREE  Report, 
Pg*e-t/T  is  used  to  determine  R  the  reliability  value.  The  value  of 
Pg,  the  probability  of  no  failures,  is  taken  as  the  numerical  measure 
ot  Reliability  R.  For  a  reliability  value  of  0.99,  e-t/f  *  e~'° 
Actually,  Pg  —  0.9900498  for  t/T  =  .010.  In  this  relation  t  is  the 
period  of  time  that  operation  is  required,  sometimes  considered  the 
time  required  to  complete  a  mission, wh l le  T  is  the  Mean-Time-Between- 
Failures  and  per  equation  <31,  p.l7t>  of  the  AGREE  report  is  f^t^/F 
where  t,  in  the  operating  time  '‘or  caoi  cciuipm  :nt  under  investi¬ 

gation  one  is  the  total  number  of  iai'.ures.  This  may  be  rev.  it  ten  as 
.1/7=0, 't^,  the  failure  tatc  lor  the  inspection  items  under  test. 

Since  this  paper  is  primarily  a  consideration  of  Reliability 
Education  and  not  a  complete  technical  d.-ve  i r.ner  t  o f  control  limits, 
a  discussion  of  possible  limits  for  the  case  above  will  illustrate 
the  nature  cf  the  material  chat  must  be  developed  and  presented  to 
reliability  engineers.  These  engineers,  in  turn,  must  understand 
these  principles  sufficiently  well  to  be  able  to  explain  these  limits 
to  management  research  and  design  engineers,  and  also  tc  production. 

Assume  that  ti  is  maintained  constant,  then  1/f  varies  with  F 
only.  _F  now  if  r.he  number  of  defects  and  for  R=0.99,  t  =  10  and  T=1000 
for  t/T--0.01C  hence  F=-001t1  and  must  be  integral  for  an  observed 

value,  i.e.  1,7,3,  etc.,  but  may  be  treated  theoretically,  “ o a L  to  non¬ 
integral  values.  Foi  tl=1000  hisv  F=l.  Treating  F  as  a  Pcisson 

distribution  then  the  standard  deviation  of  F,(Tp  =  V""F~.  hence, 

Op  =  1  =1  Tin  n  for  control  limits  including  three  times  the 

standard  deviation  about  the  expected  value  F*  =  F  +  30"p  =  F  +  3  y  F 
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and  for  F=l,  the  limits  are  -2  to  4.  Since  negative  failures  have 
no  meaning,  a  spread  of  0  to  4  includes  from  89%  to  99.73%  of  the 
distribution,  possibly  from  89%  to  99 . 865% since  the  distribution  is 
r.ot  symmetric,  but  skew.  When  F=9,  then  F*  -  F  3  f/F  ■  9  +  3(3)  -  0 
to  IS  and  an  equal  spread  about  F-9  occurs.  For~F=9,  then  t^  -  9,000 
hours. 


What  might  occur  when  *  1,000,  F«1  and  consideration  is  given 
to  an  F  value  at  the  upper  three  sigma  ( 3(f)  Control  Limit,  where  F* 
is  4.  Then  1/T  «  4/1000  *  1/250  *  .004.  For  t-10,  then  R»e-10/250  - 
e~10(  .004)  and  for  R  m  a-0.04(  R  per  p0i8s0n  Tables  is  reduced  to 
0.96,  actually  0.9607894  per  table.  The  seven  figures  given  by  the 
table  should  not  be  used  as  the  data  are  usually  not  good  for  more 
than  one  or  at  most  two  significant  figures. 

What  happens  when  t^  -  9,000  hours  and  F-9  for  a  value  at  the 
upper  control  limit,  i.e.  F*-18.  1/T*  -  18/9,000  »  .002.  For  t-10, 

then  R=e“10( .002)  m  e-0.02  —  O.gg.  Thus,  for  t^  -  1,000  a  vaiue  of 

F  at  the  30"  upper  control  limit  reduces  R  from  0.99  to  0.96.  However, 
if  ti  «  100,  then  F  =  0.10  and  F*  -  F  +  3A.JT  -0.10+3  V*  1 0  • 

0.10  +  3 ( .31023)  =  0.10  +  0,948?  «  1.0407,  For  this  value  1/T*  is 
1.05/100  -  .0105  and  R  =  e-10'-CK5>  -  e'-105  and  R  -  0.90.  Small 
samples  provide  less  reliability.  Shorter  periods  of  operation  do 
likewise.  All  these  values  were  computed  for  t  -  10  hours.  The 
table  below  tabulates  these  upper  3CT  values  of  F  for  t  values  of 
1,10,100  and  1,000  hours,  respectively  for  a  few  selected  values  of  F. 

The  probability  values  of  .89  and  .9973  noted  for  30”  limits  about  the 
expected  or  mean  value  are  obtained  from  two  assumptions.  The  .89 
comes  from  Tchebychc f f ' s  Inequality  which  reads  as  follows:  "The 
probability  that  a  variable  X  should  deviate  from  its  mean  by  more  than  X 
times  its  standard  deviation  is  equal  to  or  less  than  1A  when  k-3 
then  1/k2  =  1/9  and  8/9  «  .8888  or  .89  to  two  significant  figures. 

The  .9973  is  the  probability  value  read  from  normal  law  area  tables 
for  X  +  3CTX.  it  should  be  noted  that  no  restrictions  on  the  shape  or 
nature  of  the  distribution  are  applicable  when  using  Tchebyrhe f f ' s 
Inequality.  For  this  case  the  Camp-Meidel  extension  may  apply.  It 
reads:  "If  the  distribution  of  X  is  unimodal , . the  probability  that 

X  should  deviate  from  its  mean  by  more  than  k  times  its  standard 
deviation  is  equal  tc  cr  less  than  1/2. 25k2."  for  k-3,  then 
1/2. 25k2  =  1/20  25  or  equals  0.0494  so  that  this  Confidence  Band  may 
oe  considered  as  being  0.95  for  all  practical  purposes. 
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TABLE  1:  POSSIBLE  VARIATIONS  IN  RELIABILITY  VALUES 

CONFIDENCE  LEVEL  »  0.95  (Approximately) 
upt~i  Control  Limit  F*  -  F  +  30^  Level  Determined  for  Selected  Values 

of  F,  t  and  tj_ 

for  R  ■  Pg  ■  e“ty/T  -  e'O-OiO  ■  0.99 
where  1/T  -  T/ty  and  1/T*  -  F*/tt. 


t  - 

1 

t  - 

10 

1/T  - 

0,01. 

T  -  100 

Hrs. 

1/T  - 

0.001, 

T  -  1,000  Hrs. 

ti  * 

10 

100 

900 

1, 600 

10 

100 

1,000 

9,000 

F  - 

0.1 

1 

9 

16 

0.01 

0.1 

1 

9 

F*  • 

1.05 

4 

18 

28 

0.31 

1.05 

4 

18 

l/f*  - 

.105 

.04 

.02 

.0175 

.031 

.0105 

.004 

.002 

t/T*  - 

.105 

.04 

.02 

.0175 

U 

.105 

.04 

.02 

R  • 

.9003 

.9608 

.9802 

.9827 

.7338 

.9003 

.9608 

.9602 

t  « 

100 

t  =  1A 

000 

1/T  = 

0.0001 

,  T  =  10 

,000  Hrs. 

1/T=0. 

00001, T*100, 000  Hrs. 

M 

10 

100 

1,000 

10,000 

90,000 

10 

1,000 

100,000 

F 

0.C01 

0.01 

0.1 

1 

9 

0.0001 

0.01 

1 

F* 

.0959 

.31 

1.05 

4 

18 

.0301 

.31 

4 

1/T*  - 

.00959 

.0031 

.00105 

.0004 

.0002 

.00301 

.00031 

.00004 

t/T*  - 

.959 

.31 

.105 

.04 

.02 

3.01 

.31 

.04 

R  ■ 

.  3837 

.7338 

.9003 

.9608 

.9802 

.0493 

.7338 

.9608 

The 

table 

above 

illustrates  the 

point 

that  in 

a  true 

Reliability 

Training  Program  it  is  necessary  to  consider  all  phases  of  Reliability 
evaluations  since  the  usual  cases  given  in  articles  and  texts  are  the 
simpler  cases  given  to  illustrate  the  principles.  In  practice,  the 
cases  are  more  complicated  so  that  good  estimates  of  actual  relia¬ 
bility  attained  are  obtained  by  taking  into  consideration  all  factors. 
It  is  no  wonder  that  little  credence  is  given  to  reliability  figures 
as  now  given  since  their  error  is  necessarily  very  large.  In  many 
cases  reliability  values  are  just  guesses  since  practically  no  data 
are  available  in  the  field  covered.  In  other  coses  however,  there  is 
little  excuse  for  the  large  errors  found  in  the  estimates.  Mean  values 
must  be  supplemented  by  possible  ranges  within  which  the  actual  values 
may  fluctuate.  These  may  be  found  by  evaluating  reliability  controls 
that  reflect  both  control  and  also  lack  of  control.  Possible  worst 
cases  when  considered  carefully  might  hold  up  the  firing  of  a  missile. 
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However,  if  the  Indication  of  potential  trouble  results  in  the 
location  of  several  possible  causes  of  potential  malfunctions 
before  the  firing  and  results  in  a  success  rather  than  a  failure, 
such  reliability  evaluations  have  more  than  Justified  their  compila¬ 
tion''  '-d  evaluations. 

6 ,  Education  for  Reliability 

It  is  necessary  to  cover  all  the  different  areas  wherein  it  is 
possible  to  have  any  degradation  in  the  quality,  reliability  and 
operation  of  the  product  or  article  under  consideration.  The  article 
might  be  a  complete  missile  or  only  a  screw,  rivet  or  washer.  If  a 
lock  washer  is  net  in  its  proper  piece  end  is  ineffective,  parts  may 
not  work  due  to  vibration,  hence,  it  is  Imperative  that  attention  be 
paid  to  all  minute  details.  Hence,  one  of  the  first  elements  in  an 
educational  program  for  reliability  is  to  show  those  individuals 
responsible  for  making  an  article  the  importance  of  maintaining  speci¬ 
fication  requirements  at  every  stage  of  development  and  production  and 
particularly  for  those  elements  that  are  critical  and  also  for  those 
inaccessible  parts  difficult  to  replace  or  repair.  The  first  educa¬ 
tional  objective  for  reliability  is  the  development  of  quality- 
consciousness  . 

There  arises  another  area  where  reliability  education  is  required. 
Many  companies  are  not  certain  concerning  the  exact  demands  In  a  con¬ 
tract  or  purchase  order  concerning  its  reliability  requirements. 
Intrinsically  managements  strive  for  the  best  quality  available  for  the 
price  or  budget  money  allotted.  Sales  and  Schedules,  the  two  S’s, 
come  first  with  profit  possibly  having  even  a  higher  priority.  Design, 
engineering,  quality,  reliability,  longevity,  satisfactory  performance 
after  storage  for  long  periods  of  time  such  as  after  five  (5)  years  of 
shelf  life,  and  many  other  features  are  secondary  and  are  often  taken 
for  granted.  The  educational  program  should  place  these  in  their  proper 
perspective . 

Since  many  specifications  for  the  military  devote  50  to  BOX  of 
their  pages  to  reliability  requirements,  it  is  necessary  to  meet  these 
stipulations.  Unfortunately,  hardware,  electronic  parts,  modules  and 
units  may  be  assembled  without  too  much  difficulty  and  the  resultant 
assembly  will  usually  work  after  a  fashion.  If  parts  used  came  from 
controlled  universes  that  satisfy  the  specified  engineering  requirements, 
including  minimum  and  maximum  values  for  a  large  number  of  the  parts, 
the  resultant  products  may  be  extremely  satisfactory.  what  must  be 
emphasized  is  that  this  Is  not  always  true.  Quality  engineering  requires 
constant  vigilance.  Since  it  is  too  costly  to  perform  100X  inspections 
and  tests  time  and  again,  train  technicians  and  engineers  in  sampling 
techniques  so  that  adequate  check  or  audit  inspections  may  be  made  at 
strategic  stations.  Also  train  these  individuals  in  such  a  way  that 
they  can  select  the  strategic  stages.  This  additional  protection  is 
needed  in  Research  and  Development  contracts  particularly  at  the  time 


parts  are  being  assembled,  bench  test3  are  being  run,  environmental 
tests  are  being  conducted  or  waived,  so  that  sufficient  information 
is  available  to  insure  a  reliable  design  and  a  quality  product. 

What  is  needed  is  an  appreciation  of  integrity,  dependability,  relia¬ 
bility,  ruggedness,  ability  to  stand  shock  and  abuse,  and  what  each 
individual  can  do  to  ensure  that  the  overall  quality  and  performance 
desired  will  be  achieved.  The  second  objective  in  reliability  educa¬ 
tion  is  to  train  individuals  to  differentiate  between  minor  and 
critical  details  and  use  their  efforts  most  effectively  in  the  areas 
of  greatest  importance.  Tersely  this  may  be  stated  es  placing  first 
things  first  to  obtain  maximum  reliability. 

A  third  objective  must  be  stressed.  Training  must  be  given  in 
modern  methods  and  techniques  for  reliability  evaluation.  There  must 
be  developed  an  appreciation  of  the  limitations  of  certain  parts  of 
the  theory  to  date  and  the  necessity  of  doing  research  in  reliability 
theory  and  allied  areas  such  as  operations  research,  mathematical 
statistics  and  also  basic  physics.  Reliability  engineers  working  in 
electronics  must  realize  that  much  of  their  data  is  subject  to  change 
and  re-evaluation,  being  based  on  relatively  small  samples.  Environ¬ 
ments  are  changing  and  the  demands  of  service,  particularly  the  military 
are  much  more  exacting.  Errors  of  measurement  that  formerly  could  be 
ignored  must  now  be  considered.  Hence,  the  third  objective  iB  advanced 
technical  training  in  reliability  methodology. 

7 .  Economic  Factors  -  Standards,  Value  Engineering,  Interchange  of 
Data  and  Reliability 

In  Reliability  Education,  use  must  ba  made  of  all  past  available 
material,  methods  and  techniques.  The  truly  effective  Reliability 
Engineer  must  not  only  know  the  product  being  considered  better  than 
those  in  charge  of  the  product  but  also  must  be  acquainted  with  the 
most  recent  engineering  advances  in  all  fields.  Hence,  his  design 
evaluations  will  be  accepted  as  being  authentic,  valuable  and  a  con¬ 
tribution  to  the  project.  He  must  be  acquainted  with  critical  tests 
so  that  he  can  take  charge  of  the  most  modern  test  facilities  and 
conduct  tests  that  are  meaningful.  Since  it  is  impossible  to  obtain 
all  these  qualities  in  one  man,  then  a  director  plus  experts  in  the 
areas  under  consideration  will  make  up  a  valuable  reliability  team. 

The  approach  is  similar  to  that  used  in  Operations  Research  based  on 
the  handling  of  problems  by  properly  chosen  teams. 


Standards  must  be  established  that  are  economic.  They  must  be 
in  line  with  market  demands  and  prices.  It  is  necessary  to  work  with 
Value  Engineering  groups  In  properly  evaluating  the  selection  of 
proper  designs  and  component  parts.  Evaluating  complaints  on  similar 
equipments,  establishing  the  level  of  customer  demands,  setting  up 
procedures  and  tests  for  evaluating  component  parts  -  all  are  a  part 
of  the  prcpct  training  in  Reliability.  Establishing  excellent  labora¬ 
tories  for  obtaining  controlled  data  is  a  must  since  there  must  be 


established  more  economical  means  for  evaluating  component  parts 
obtained  from  varied  sources.  Specifications  and  procedures  as 
well  as  designs  are  written  around  "hardware”.  This  means  that 
the  characteristics  of  this  "hardware"  commonly  used  for  military 
pr nt?:'*-;  must  be  evaluated,  tabulated,  and  be  available  to  all 
con tractors. 

«  Attempts  have  beer,  made  to  secure  some  central  agency  to  main¬ 

tain  such  files.  With  proper  publicity  and  broader  training  programs 
in  reliability  an  appreciation  of  these  needs  will  eventually  be  so 
apparent  that  steps  will  be  taken  to  achieve  this  goal.  A  start  has 
been  made  through  interchange  of  data  on  very  similar  projects  ln- 
«  volving  missiles  at  this  tims.  Training  engineers  to  carry  out  this 

task  in  such  form  as  to  be  most  effective  will  forward  the  reliability 
program  and  allow  more  time  for  new  evaluations,  new  researches  and 
also  release  funds  for  audit  inspections  and  tests  as  well  as  better 
design  evaluations.  With  much  more  basic  data  available  to  the 
designers  and  the  engineers,  their  designs  should  improve  in  many 
areas  and  in  particular  in  the  field  of  Reliability. 

Since  this  Section  is  devoted  to  the  Economic  Factors  with  respect 
to  Reliability,  it  is  necessary  to  reconsider  the  problem  as  a  whole  and 
see  what  steps  were  taken  in  the  early  stages  of  Quality  Engineering  to 
achieve  Reliability.  Reliability  Education  must  consider  not  only  new 
knowledge  but  past  history  with  respect  to  the  basic  factors  of 
Reliability,  kindred  or  allied  areas  of  activity,  and  what  Reliability 
was  achieved  in  the  past  by  these  other  methods  and  techniques  used 
prior  to  the  cry  and  contracts  for  Reliability. 

In  1926  the  Bell  System  had  Committees  on  Inspection  and  Rating  of 
the  Quality  of  Manufactured  Products  operating  and  holding  regular 
meetings  about  every  two  or  three  months.  The  Rating  Committee  was 
concerned  w  th  Reference  or  Standard  Quality  Levels,  Control  Charts  for 
maintaining  control  at  these  levels,  and  methods  for  taking  into 
consideration  the  complexity  of  the  equipment  and  systems,  but  also  the 
seriousness  of  the  requirements.  Economic  Standards  of  Quality  were 
established  as  Standard  Demerits  per  Unit  values.  These  levels  were 
based  on  Manufacturing  performance,  but  were  modified  to  take  account  of 
costs,  malfunctions,  field  reports,  and  actual  complaints  received  from 
the  Telephone  Companies  using  the  Equipments  and  Apparatus. 

Many  will  not  agree  that  such  a  Rating  System  is  related  to  the 
evaluation  of  Reliability.  The  results  were  tied  in  with  other  results 
and  one  series  of  charts  was  developed  that  presented  Troubles  per 
.  Trouble  Unit.  Many  of  the  charts  were  established  so  that  with  the 

current  Check  Inspection  or  Audit  Inspection  samples  being  used,  the 
number  in  such  samples  was  such  that  with  the  current  standards  of 
Demerlts-Per-Unit  values  and  the  standard  Constant  C„  used  to  determine 
the  control  limits,  based  on  sound  Statistical  Theory,  the  occurrence  of 
*  a  single  Critical  defect  with  a  Major  classification  would  throw  the 
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Dy  ( Demerits-Per-Unlt)  Value  outside  control  limits,  where  a 

minimum  monthly  sample  site  was  given.  Defects  or  Nonconformances 
really  Critical,  are  generally  always  inspected  100%.  Major  A' a 
and  B's  with  Demerit  weights  respectively  for  each  item  of  100  and 
SO  rivm-.its  were  established  so  that  effective  standards  and  controls 
over  in-process  and  final  products  may  be  achieved.  Minor  C's  and 
D's  had  assigned  to  them  respectively  demerit  values  of  10  and  1. 

This  Rating  System  was  tried  out  on  Jet  Engines  by  General 
Electric  and  appears  to  have  been  found  very  satisfactory.  It  has  been 
applied  to  Repair  operations  in  the  field  by  the  Air  Force.  The  Bell 
System  uses  it  in  all  their  various  divisions  and  activities,  including 
their  29  Supply  Depots  scattered  over  the  nation.  Missile  and  Aircraft 
Manufacturers  have  been  against  this  system  as  they  were  afraid  that  it 
would  be  too  complicated.  Very  few  companies  tried  the  right  approach 
in  setting  up  Demerit  Lists  for  the  characteristics  and  inspection 
it^ms  applicable.  These  can  readily  be  grouped.  Setting  up  individ¬ 
ual  values  for  each  characteristic  would  be  too  costly.  As  a 
compromise,  using  the  group  method  it  is  possible  to  establish  sample 
sizes,  criteria,  courses  of  action,  methods  of  presentation,  etc.,  to 
such  an  extent  that  all  items  used  in  quality  and  reliability  evalua¬ 
tion  are  covered  by  sample  results.  Rating  Charts  may  Judiciously  be 
used  as  a  Reliability  Index  if  arranged  in  Index  Form.  The  mathema¬ 
tical  relations  are  given  in  the  next  paragraph. 

The  most  general  case  occurs  when  each  characteristic  or 
inspection  item  is  considered  separately  so  that  the  demerits  per-unit 
value,  is  obtained  from  the  relations 

Du  "  2>iV"i  '  (1) 

where  Du  *  Demerits  -  per-unit, 

w^  ■  Demerit  value  assigned  to  inspection  item  i, 

■  Observed  number  of  defects  for  the  1th  inspection  item 
in  a  sample  of  ni  units, 

=  Sample  size  for  the  ith  inspection  item,  and 

Inspection  Item  »  Any  characteristic,  property  or  requirement 
for  which  inspection  (including  test)  is 
requl red. 

Now,  it  Is  easier  to  have  the  same  sample  size  for  a  group  of  items 
where  a  four-fold  classification  of  defects  for  other  than  critical 
items  is  used,  l.e.  for  major  and  minor  defects,  and  a  sample  of  size 
nA  is  used  for  all  Class  A  defects  or  inspection  items,  nB  for  Class  B 
defects  or  inspection  items,  n^  for  Class  C  defects  or  inspection  items 
and  nn  is  the  sample  size  for  inspecting  the  Class  D  defects  or  inspec¬ 
tion  items.  This  reduces  equation  (1)  to 

Du  "  wAdA/nA  +  wBdB/nB  +  wcdc/nC  +  wDdD/^  nD  -  <  2> 
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and  is 


The  standard  or  expected  value  of  Du  is  written  as  DUa 

derived  either  from  past  history  or  from  an  analysis  of  customer 
demands  and  desires.  Then  the  following  standards  are  established 
for  each  of  the  four  groupst 

PA3  -H<VnA'  PS8  •  PC8  PDs  (3) 

where  ^nA,  nB<  nc*  *■  ^nD  are  th®  sums  of  samples  taken 

uring  a  representative  period  of  production  or  development.  Usually 
n  •  nA  ■  nB  ■  nc  -  nD  for  ease  of  administration. 

I  Control  Charts  for  these  Du  values  may  be  determined  from  the 

general  relation  for  D  « 

■ 

°U8  ■  waPa8  +  wbPb8  +  wcPc8  +  wdPd8  •  (4) 

From  the  theory  of  errors  by  taking  successive  partial  derivatives 
the  standard  deviation  for  relation  (4)  is 

\  A2PA8  +  WB2PB8  +  WC2  PC8  +  wD2PD8>/n  "  ^./n  #  (5) 

Thus  the  Standard  Constant  C8  takes  the  place  of  p  in  determining 
standard  deviation  and  control  limit  values.  The  relation  for  Cs  is 

Ca  “  wA2Pa  +  WB2pB  +  WC2pC  +  WD2pD  *  ^Iwi2Pi'  U«A.B,C,D)  ,  (6) 

where  these  p  values  are  expected  values.  Control  Limits  may  be 
written  as 

%  1  % “  Dus 1  (7) 

where  t  •  3  for  staff  purposes  and  t  ■  2  for  production  purposes. 

The  Confidence  Levels  are  from  0.89  to  0.9973  for  t  ■  3  and  from 
0.75  to  .9545  for  t  =  2.  Other  forms  are  written  as  follows i 

Demerits  :  Dg  + ■  t  0^  =  Dg  +  t  *\J Cgn  ;  (8) 

where  Dg  -  n  Du  «  expected  demerits  in  sample  of  size  n. 

Index:  I  =■  Du/Du  >  (9) 

s 

•  Js  ±  t  =  1  ±  t^Cg/n'  /Du  .  (10) 

It  must  be  noted  In  our  Reliability  Fducational  Training  program 

that  the  reliability  desired  is  perfection,  100%  satisfactory  perfor- 
mance .  This  means  no  defects,  no  demerits,  ro  failures,  no  malfunc¬ 
tions  and  perfect  performance.  This  cannot  be  realized  as  we  have 
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Blight  deviations  and  also  wearout.  Hence,  the  controls  for 
Reliability  must  be  similar  to  those  above  for  the  index  where  we 
set  a  reliability  goal  as  an  Ra  and  then  determine  how  closely  we 
approach  thla  goal.  Thus  IR  -  R/Rs  and 


and  <TIr  -  <rR/R9  ,  (12) 

with  control  limits  of 

IR  -  1,  IR  +  t  <fz  -  1  +  t  CTr/R8  .  (13) 

S  S  R 

The  rating  system  used  for  many  years  is  practically  equivalent 
to  these  measures  of  reliability.  Use  may  be  made  of  a  lot  of  past 
data  and  training  material  when  developing  a  comprehensive  training 
program  for  Reliability. 

8.  Reliability  Training  Programs  -  Modern  Techniques 


Summarizing  the  above  thoughts  on  Reliability  Education,  it  must 
be  noted  that  the  most  important  element  is  to  know  when  constructive 
action  must  be  taken.  A  hlntorira1  treatment  of  reliability  can  be 
made  when  the  last  unit  of  any  type  has  been  destroyed  or  has  worn 
out.  Then  the  true  reliability  of  the  units  in  that  group  may  be 
observed.  Modern  demands  are  for  accurate  estimates  of  reliabilities 
that  will  be  obtained.  How  to  make  valid  Reliability  forecasts  is  one 
phase  of  Reliability  that  must  be  covered. 

when  working  in  Reliability  and  Quality  Engineering  what  action 
must  be  taken  to  guide  designs  into  channels  where  available  data 
indicate  the  optimum  reliability  may  be  obtained.  Operations  Research 
deals  with  the  optimization  problems  of  management.  Hence,  Reliability 
Education  requires  a  knowledge  of  Operations  Research  techniques. 

Systems  ore  beinq  developed  from  new  components.  Many  new 
articles  and  parts  are  being  considered  in  these  system  designs.  Two 
objectives  must  be  considered  here  in  an  educational  program.  One  is 
a  knowledge  of  System  Engineering  and  liaison  work.  The  second  is 
knowledge  of  component  part  reliability  values.  The  first  is  a 
philosophy  evaluating  the  whole  from  its  parts  and.  needs  to  consider 
management  philosophies  and  also  statistical  checks  such  as  those 
embodied  in  the  Analysis  of  Variance.  The  second  requires  laboratory 
test  results  with  checks  for  significant  differences  between  different 
parts  and  their  characteristics.  Management  Science  and  Advanced 
Statistical  Techniques,  even  possibly  Decision  Functions,  need  to  be 
covered  in  a  complete  Reliability  Educational  Program. 
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Re ii abi 1 i  ty  evaluation?,  point  seta,  theory  of  numbers  and 
related  techniques  need  to  be  covered  to  some  extent.  Basic 
statistical  techniques  ate  used  to  establish  the  various  probabili¬ 
ties  that  need  consideration  such  as  existence  probabilities  and 
the  like.  A  part  of  these  methodologies  include  distribution  theories. 
Monte  carlo  methods,  use  of  Poisson  exponentials,  binomials,  Normal 
law  theory.  Lexis,  Poisson  and  Bernoulli  sampling  and  their 
respective  errors.  Not  all  apply  but  those  parts  that  do  may  prove 
to  be  very  profitable  Indicating  where  reliability  may  be  obtained 
at  minimum  costs. 

Thus,  Reliability  Education  must  concern  Itself  withi 

1.  Valid  forecasts  of  Reliability  -  techniques  and  errors. 

2.  Optimizing  designs  -  techniques  of  Operations  Research. 

3.  Component  part  reliability  values  -  tests  for  significance. 

4.  Systems  engineering  -  management  science. 

5.  Distribution  and  probability  theory. 

6.  Design  reviews  -  value  engineering. 

7.  Reliability  block  diagrams  -  flow  charts. 

8.  Nonstandard  parts  -  establishment  of  economic  standards  of 
quality  for  standard  and  nonstandard  parts. 

9.  Laboratory  tests  and  evaluations. 

10.  Life  tests,  reliability  tests,  longevity,  storage  life. 

Many  others  could  be  noted.  These  will  cover  the  rudiments  of 
Reliability. 
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DEVELOPMENT  OF  A  RELIABILITY  COURSE 
SPECIFICALLY  FOR  DESIGN  ENGINEERS 

T ..  H.  Dee,  B.  D.  Hatch,  Defense  Systems  Dept.,  Syracuse,  N.  Y. 
E.  G.  Bianco,  General  Engineering  Laboratory,  Schenec tady,  N.  Y. 

General  Electric  Company 


SUMMARY 

Most  reliability  training  courses  have  been  directed  to  reliability 
engineers,  manufacturing  personnel  and  management.  Such  courses  are 
broad  in  scope  and  cover  all  phases  of  a  reliability  program.  Improving 
intrinsic  reliability,  however,  is  the  design  engineers'  problem,  and  a 
need  exists  for  a  reliability  course  specifically  aimed  at  helping  him  to 
improve  his  competence. 

This  paper  describes  the  development  of  such  a  course  for  use  in 
the  Defense  Electronics  Division  of  the  General  Electric  Company. 

INTRODUCTION 


For  several  years,  those  operations  of  the  General  Electric 
Company  which  serve  the  Department  of  Defense  have  had  in  place  organ¬ 
izational  functions  devoted  exclusively  to  reliability  improvement. 

These  reliability  groups  have  been  working  toward  their  objectives 
by  such  means  as  apportionment,  evaluation  and  selection  of  component 
parts,  tolerance  analysis  of  circuit  design,  mechanical  and  electrical 
design  reviews,  manufacturing  quality  control,  environmental  testing  and 
field  reliability  measurement,  analysis  and  feedback. 

From  these  activities  considerable  experience  has  been  gained  in 
the  techniques  for  improving  reliability  as  a  result  of  participation  in  all 
phases  of  system  flow  or  evolution,  from  the  specification  through  design, 
manufacture,  shipment,  and  installation,  to  Field  use. 

This  system  flow,  however,  spans  a  considerable  period  of  time, 
up  to  several  years  on  complicated  systems,  and  means  that  the  function¬ 
ing  of  a  reliability  group  is  spread  out  in  time  as  well  as  being  spread 
over  all  the  established  functions  such  as  engineering,  manufacturing, 
field  operations,  etc. 
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These  two  factors,  tine  span  and  functional  span,  arc  significant 
reasons  for  the  difficulties  encountered  in  establishing  the  authority  of  a 
reliability  group  in  any  organizational  structure. 

In  the  areas  of  defining  and  specifying  reliability  programs, 
significant  progress  has  been  made  by  governmental  agencies,  an  exam¬ 
ple  being  AFBM  Exhibit  58-10;  entitled,  "Reliability  Program  for 
Ballistic  Missile  and  Space  Systems".  However,  when  a  reliability 
program  is  to  be  implemented,  it  is  still  necessary  to  determine  a  clear 
concept  of  those  specific  activities  of  the  organization  to  which  the 
reliability  group  can  make  effective  contributions. 

A  review  of  the  responsibilities  and  authorities  of  all  functions  in 
the  total  system  flow  of  an  organization  reveals  that  each  function,  with 
the  exception  of  design  engineering,  has  a  responsibility  for  reliability 
which  is  limited  by  the  engineering  drawings  and  other  documentation. 

In  other  words,  these  other  functions  cannot  produce,  install  or  operate 
a  system  that  is  any  better  than  the  design. 

This  may  be  considered  to  be  just  another  way  of  Baying  that 
reliability  is  inherent  in  design,  but  it  does  serve  to  emphasize  that  there 
are  basic,  fundamental  limitations  to  the  improvement  in  reliability  that 
can  be  expected  or  achieved  in  functions  other  than  engineering. 

There  is  good  logic  in  this  situation,  in  that  the  system  and  equip¬ 
ment  are  first  created  in  detail  when  the  engineering  function  has  com¬ 
pleted  the  design.  Then,  with  drawings  and  other  engineering  documen¬ 
tation  passed  to  the  manufacturing  and  succeeding  functions,  there  exist 
the  detailed  criteria  against  which  the  output  of  each  function  can  be 
measured,  as  by  Quality  Control,  Inspection  and  Test.  These  functions 
cannot  be  required  or  expected  to  provide  output  that  is  any  better  than 
their  input,  which  has  born  spelled  out  by  the  engineering  function. 

Evaluation  o(  an  engineering  function  is  difficult  to  measure  be¬ 
cause  its  inputs  arc  in  the  form  of  general  and  functional  requirements. 
On  the  other  hand,  output  cannot  be  measured  until  something  is  created 
--the  detailed  design.  In  the  creative  process,  engineering  must  origi¬ 
nate  the  detailed  design  and  build  breadboard  and  prototype  models  by 
which  the  function  run  be  measured.  Even  then,  final  measure  cannot  be 
made  until  the  system  flow  has  progressed  to  the  operational  phase. 
Consequently,  ltd  m  bi  I  ity  iniprovcimjnt  and  the  development  of  techniques 
to  achieve  it  must  be  r  or, rent  rail'd  at  the  early  design  stage. 
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These  are  reasons  why  reliability  groups  are  generally  in  the 
engineering  function,  and  why  the  strongest  reliability  efforts  should  be 
devoted  to  the  early  design  phase.  There,  we  have  the  earliest  possible 
or.o^’-tunity  in  the  time  flow  of  a  system  to  improve  its  inherent 
reliability,  and  can  operate  most  effectively  to  fulfill  realistic  engineer* 
ing  responsibilities. 

Reliability  groups  perform  many  line  and  staff  services  in  the 
engineering  function.  An  additional  service  is  to  improve  the  compe¬ 
tence  of  the  engineers  themselves.  This  is  accomplished  by  individual 
counseling,  design  reviews,  publications,  and  more  recently  in  our  own 
Defense  Electronics  Division,  by  an  engineering  course  specifically 
prepared  for  design  engineers. 

Development  of  the  Engineering  Course,  "Reliability  by  Design" 

The  introduction  emphasized  the  amount  of  consideration  that 
was  given  in  the  development  of  the  course  to  the  extraction  of  specific 
work  activities  from  the  philosophical  concept  that  "reliability  is  every¬ 
body's  business."  These  are  the  areas  in  which  reliability  groups  can 
accomplish  the  meat  good. 

The  development  of  the  course  started  with  the  preparation  of  an 
outline  by  a  task  force.  Since  it  had  been  decided  that  the  student  body 
should  consist  principally  of  design  engineers,  it  was  necessary  to 
focus  attention  on  those  subjects  which  would  be  of  use  to  them,  and  to 
resist  the  tendency  to  include  those  which  are  generally  included  in  an 
overall  reliability  program. 

Then,  recognizing  that  "Reliability"  may  be  considered 
"Performance  as  a  function  of  time"  it  was  decided  that  the  objective  of 
the  course  rnust  be  to  augment  the  design  engineer's  knowledge  of  how 
to  achieve  specified  performance  with  expanded  knowledge  of  how  time 
and  environment  affect  materials,  parts  and  circuit  functions. 

Also,  it  was  decided  that  we  would  not  spend  too  much  early 
course  time  on  the  mathematics  of  probabilities,  which  might  mislead 
the  engineer  into  believing  that  reliability  is  statistics  rather  than 
engineering . 
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Instead,  the  outline  was  developed  with  a  concentration  on  the 
basic  engineering  technologies,  which  are  the  ultimate  source  of  desig.i 
for  higher  reliability,  and  the  mathematics  was  introduced  as  needed. 

O  '  ■>  of  the  Course 


The  course  sessions  were  held  weekly  for  two  class  groups, 
and  lasted  about  2-1/2  hours  each.  One  group  met  in  the  morning,  and 
the  other  in  the  afternoon,  on  Mondays.  Several  reliability  engineers 
from  General  Electric  plants  in  other  cities  also  attended  for  the 
purpose  of  applying  the  material  to  reliability  courses  at  their  locations. 


The  outline  of  the  course  as  it  was  conducted  this  past  Fall  term 
of  16  sessions  Is  as  follows: 


Session 

1 

2 

3 

4 

5 

6 

7 

8 


Subj 


ec  t 


Introduc  tion 
Probability  Concepts 
Reliability  Design  Principles 
Mechanical  Design 
Heat  Transfer 
Nuclear  Radiation 
Reaction  Kinetics 
Dielectric  Behavior 


s* 

s 


,  *  _ 

v  r""" 


10,11,  and  12 

13 

14 

15  and  16 


Predicting  Reliability 
Mathematics  of  Reliability 
Reliability  Measurements 
Design  of  Reliability  Tests 
Design  Case  Histories 
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With  the  exception  of  the  case  history  sessions,  the  course  was 
written  and  conducted  by  eight  consulting  engineers  of  the  General 
Engineering  Laboratory,  which  is  a  service  operation  for  the  entire 
General  Electric  Company.  The  case  history  sessions  were  presented 
by  engineering  personnel  of  the  Defense  Electronics  Division. 

The  homework  consisted  principally  of  reading  the  lecture  notes, 
a  total  of  about  450  pages,  which  were  distributed  prior  to  the  class 
ses  sions . 

At  this  point  a  summary  of  each  of  the  sessions  may  be  of 
interest. 

Session  1  Introduction 

Two  managers  of  reliability  groups,  Messrs*  G.  M.  Armour 
and  B.  D.  Hatch,  who  have  developed  and  directed  reliability  activities 
in  the  Defense  Electronics  Division  for  a  number  of  years,  described 
the  history,  problems,  accomplishments  and.  challenges  of  this  rela¬ 
tively  new  field  of  engineering. 

Mr.  E.  G.  Bianco,  coordinator  of  the  course  development  in  the 
General  Engineering  Laboratory  and  also  a  lecturer,  presented  the 
outline  and  objectives  of  the  course. 

Session  2  Probability  Concepts 

This  session  was  a  fundamental  exposition  of  probability  as  it 
applies  to  the  basic  reliability  tenets.  It  was  intended  to  serve  as  a 
precursor  to  such  concepts  as  prediction,  analysis  and  redundancy. 
Much  of  the  material  covered  here  was  expanded  and  illustrated  in 
subsequent  lectures.  The  format  followed  that  of  a  basic  course  in 
probability,  including 

Defini  lion  of  Probability 

Probability  Rules  Given: 

-  Mutual  Exclusion 

-  Independence 


■)  -  1# 


Conditional  Probability 


Bernoulli  Trials  and  the  Binomial  Model 

-  A  Basic  Method  for  Acceptance  Testing 

Poisson  Behavior  and  the  Exponential  Law 

Session  3  Reliability  Design  Principles 

A  detailed  program  plan  was  presented  for  the  design  engineer, 
covering  the  product  flow  from  specification  to  field  data  feedback. 
Some  of  the  principal  steps  included  were: 

Requirements 

Preliminary  Design 

Parts  Selection 

Estimation  of  Reliability 

Design  Centering 

Tolerance  Analysis 

Drift  Effects  and  Aging 

Design  Reviews 

Test  Planning 

Quality  Control  Requirements 
Testing  and  Analysis 
Field  Operation  Analysis 
Design  Improvement 


I 


i 
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Session  4  Mechanical  Design 


The  mechanisms  of  failure  in  mechanical  design  and  the  means 
for  testing  and  designing  to  avoid  them  were  the  subjects  of  this 
_^«sion.  Emphasis  was  on  the  analysic  of  the  probability  of  failure, 
and  accelerated  life  tests  and  evaluation  methods. 


Some  of  the  subject  headings  were: 
The  Design  Problem 

-  Types  of  Failure 

-  Manufacturing  Tolerances 

-  Quality  Control 

-  Design  Philosophy 


-  Fail  Safe 


-  Redundancy 


"One -horse -shay" 


Environment 


-  Loads 


-  Temperature 


-  Corrosion 


Stress  Analysis 


Material  Behavior 


Laboratory  Tests  in  Designing  for  Fatigue 
Test  Analysis  Techniques 
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Session  5  Heat  T ransfer 

This  session  was  concerned  with  the  techniques  of  designing  for 
control  of  temperature,  in  order  to  avoid  reliability  degradation  and 
fauure.  This  involves  heat  generation,  dissipation,  and  temperature 
rise,  and  specific  subjects  included: 

Failure  Rate  as  a  Function  of  Temperature 

Sources  of  Heat 

Modes  of  Heat  Transfer 

-  Convection,  Free  and  Forced 

-  Radiation 

-  Conduction 

-  Evaporation  and  Boiling 
Temperature  Rise 
Temperature  Distributions 
Arrangement  of  Parts  and  Enclosures 
Heat  Sinks 
Measurements 

Session  6  Nuclear  Radiation 

The  several  kinds  of  radiation  from  nuclear  materials  and 
explosions,  and  their  effects  on  electronic  equipment,  both  instantane¬ 
ous  and  cumulative  were  presented.  Some  of  the  subject  headings  were: 

Ty|K.-H  of  Radiation 

Inter, iction  with  Matter 


Effects  on  Materials 


Organic 


Inorganic 


Effects  on  Electronic  Components 


Designing  for  Radiation  Service 


Radiation  Testing 


Effects  on  Circuit  Operation  of  High  Dose  Rates 


-  Recovery  Rates 


Session  7  Reaction  Kinetics 


This  session  dealt  with  the  fundamental  causes  of  failures  in 
terms  of  the  reactions  between  materials  which  alter  their  molecular 
and  atomic  structures.  The  subject  headings  included: 


What  Materials  May  Interact 


Effects  of  Temperature  and  Concentration 


Reaction  Speed 


Effects  of  Catalys's 


Reactions  in  Enclosed  Modules 


Irradiation  Effects  on  Reaction 


Corrosion  as  a  Reaction 


Effects  of  Reaction  on  Mechanical  Failure 


Session  8  Dielectric  lU-havior 


The  mechanisms  and  causes  of  degradation  and  failure  in 
dielectric  materials,  and  what  can  be  done  to  evaluate  them  in  designing 
for  reliability  were  the  objectives  of  this  session.  Specifically  con¬ 
sidered  wore: 
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Mechanism  of  Gaseous  Breakdown 
Corona 

Mechanism  of  Breakdown  in  Liquids  and  Solids 

The  Effects  of  Voids  in  Dielectrics 

Differences  Between  a-c  and  d-c  Test  Procedures 

Insulation  Testing  for  Moisture,  Polarizable  Contamina 
tion,  etc. 

Session  9  Predictino  Reliabilit 


In  this  session  there  was  described  the  generally  accepted 
techniques  for  predicting  the  reliability  of  equipment  with  the  expon¬ 
ential  or  constant  failure  rate  process  assumed.  A  thorough  critique  of 
these  techniques  was  presented.  In  addition,  the  effects  of  other  failure 
rate  processes  were  reviewed.  Subject  headings  included: 

The  Exponential  Case,  Assumptions  and  Limitations 

Analytical  Procedures 

Reported  Failure  Rate  Data 

Random  or  Chance  Failures 

Secondary  Failures 

Early-Life  Failures 

Wear-Out  Failures 

Degradation  Failures 

Non- Exponential  Failure  Distributions 


Derating  of  Parts 


Iii  these  sessions  a  mathematical  treatment  of  failure  processes 
v.-3<5  developed.  Principle  failure  mechanisms  were  illustrated  and 
derived  from  stochastic  models.  Particular  emphasis  was  given  to 
methods  of  estimation  and  evaluation  of  life  behavior  parameters  from 
test  data.  The  subjects  covered  were: 

Derivation  of  the  Poisson  Process 

Generalisation  of  Poisson  Behavior  to  Weibull,  Gumbel , 
Gamma  and  "Pareto"  models 

Estimation  Theory 

Conditional  Failure  Density  and  Failure  Rate  Structures 
Techniques  for  Analysis  of  Failure  Data 

-  Tests  for  Appropriateness  of  Model 

-  Methods  of  Estimation  for  a  Particular  Model 

-  Significance  Tests 
Confidence  Statements 

Sessions  1  3  and  14  Reliability  Measurements  and  Tests 


These  sessions  extended  the  earlier  developed  theory  and  tech¬ 
nique  into  the  realm  of  acceptance  and  estimation  life  testing.  Careful 
distinctions  were  noted  between  the  two  types  of.test  approaches. 
Detailec  attention  was  given  to  aspects  of  the  accelerated  test  rationale 
in  which  both  coinpressed-time  testing  and  increased-severity  testing 
were  explained.  Further  emphasis  was  given  to  new  techniques  in  this 
field  e,g.  Life  "barometer"  studies;  life-stress  relationships  via 
response  surfaces,  etc.  The  principal  s ubjec t  headings  included: 

Definition  of  Product  Life 


Life  Testing  Theory 


’-.vAdbt\  N  • 


Results  of  the  First  Run  of  the  Course 


The  reaction  of  the  engineers  taking  the  course  has  been 
^-T-.r  raiiv  Rood.  However,  a  criticism  has  been  that  using  ten  lecturers, 
each  expert  in  his  own  field,  has  resulted  in  some  lack  of  unity  or  inte¬ 
gration  of  the  material  presented.  Also,  there  was  not  sufficient 
"cook-book"  information  to  satisfy  some  of  the  engineers. 

The  Spring  term  sessions  of  the  course  this  year  will  endeavor 
to  alleviate  these  conditions  by  having  a  single  instructor  conduct  most 
of  the  sessions,  and  integrate  more  "cook-book"  procedures  in  the  flow 
of  the  course.  However,  those  sessions  which  are  particularly  oriented 
to  single  technological  areas  will  again  be  presented  by  specialists. 

The  basic  lecture  notes  as  they  now  exist  (about  450  pages)  will 
be  used  as  reference  material  and  will  be  supplemented  to  reflect  tne 
changes  and  additions  to  the  course. 

Other  departments  of  General  Electric  in  other  citieB  are  also 
presenting  reliability  courses.  These  are  being  developed  for  the 
particular  needs  of  those  departments,  many  of  which  are  in  the  indus¬ 
trial  and  commercial  fields,  and  will  utilize  portions  of  the  subject 
course  as  they  are  applicable. 
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INCREASING  RELIABILITY  THROUGH 
OPERATOR  AND  INSPECTOR  TRAINING  AND  CERTIFICATION 

Robert  F.  Koenig,  Manager,  Materials  and  Processes  Engineering 
General  Electric  Conpany 
Missile  and  Space  Vehicle  Department 

Substantial  amounts  of  money,  and  even  more  important  -  time,  are  being 
saved  in  our  re-entry  nd  space  vehicle  development  programs  bv  an  ext¬ 
ensive  program  of  training  and  certifying  operators  and  inspectors. 
Reliability  is  also  being  increased  thru  reduction  of  operator  variab¬ 
ility  and  improved  process  inspection. 

The  training  and  certification  of  operators  and  inspectors  is  not  new. 
For  instance,  weldors  (1)  and  penetrant  inspectors  (2)  require  certi¬ 
fication  to  work  on  military  equipment  (even  though  such  a  requirement 
is  contrary  to  the  generally-accepted  practice  of  specifying  results 
instead  of  methods  for  achieving  them).  One  factor  which  undoubtedly 
led  to  the  weldor  certification  requirement  was  the  wide  variation 
observed  in  the  interpretation  and  application  of  standard  welding 
procedures  by  individual  operators,  A  second  factor  was  the  difficulty 
in  some  cases  of  determining  weld  quality  on  complete'!  airframes  and 
structures. 

What  is  different  about  the  training  and  certifying  of  operators  and 
inspectors  is  the  extent  and  depth  of  the  program  at  G.E.'s  Missile  and 
Space  Vehicle  Department.  The  delays  and  cost  of  rework  observed  as 
our  reentry  vehicle  program  entered  initial  production  stages  were 
great  enough  to  justify  investigation  and  corrective  action.  Inspection 
reports  and  failure  analyses  showed  that  many  rejections  and  failures 
were  directly  attributable  to  operator  technique,  inspector  knowledge, 
or  both.  Consideration  of  this  situation  led  to  the  conclusion  that  one 
of  the  characteristics  of  today's  R.  &  D.  business  -  the  absence  of 
sufficient  repetitive  production  for  the  development  of  high  degrees  of 
proficiency  -  was  a  significant  factor.  Even  though  the  same  basic 
process  may  be  used  in  a  series  of  models,  there  was  usually  enough 
difference  from  model  to  model  to  require  variations  in  technique.  One 
way  around  this  was  to  give  the  operators  and  inspectors  some  under¬ 
standing  of  vh.it  they  were  doing  and  the  reasons  for  the  steps  in 
standard  processes  so  that  they  would  be  better  able  to  cope  with  a 
variety  of  applications  of  the  same  basic  processes. 


(1)  M1L-T-5021  ,  Ti  :,t  ,  Aircraft  Welding  Operators'  Certification 

(2)  MIL -STD-4 K-,  C ■  r tl f 1 cat  1  on  of  Penetrant  Inspection  Personnel 
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A  second  factor  In  the  decision  to  provide  a  training  and  certification 
program  van  the  rapidity  with  which  new  and  relatively  unevaluated 
processes  were  introduced  into  the  production  cycle.  Metal-to-*etal 
and  metal-to-plastlc  Joint*  are  being  bonded  routinely  today  when  only 
•  f 'v  years  ago  this  waa  considered  an  experimental  achievement.  In 
many  caaea  the  process  waa  every  bit  as  critical  and  just  aa  subject  to 
variations  as  a  result  of  operator  performance  as  welding.  Why  should 
any  leas  care  be  exercised  la  selecting  personnel  to  bond  critical 
assemblies  together  than  we  would  use  in  selecting  weldors?  The  same 
is  true  for  solderers  -  in  missile  systems,  soldered  joints  can  be  Juat 
aa  critical  as  weldments. 

These  considerations  led  to  the  decision  to  adopt  training  and  certi¬ 
fication  program  for  the  following  processes t 

Manufacturing  Processes 

Alodlnlng 
Bonding 
Foaming 
Heat  Treating 
Lay-Up  Laminating 
Passive ting 
Potting 
Soldering 
Spot  Welding 
Spray  Painting 
Torch  Brazing 

Mew  courses  are  added  as  new  processes  are  introduced  in  the  shops. 

With  each  course,  the  same  general  procedure  la  followed.  The  candidates 
for  quail fleet ion  attend  a  period  of  one  or  more  hours  of  class  room 
instruction  in  the  basic  theory  behind  each  step  in  the  operation 
procedure.  Abbreviated  texts  covering  the  subject  matter  presented  are 
given  to  each  operator.  Following  that,  they  are  taken  into  the  labor¬ 
atory  where  they  observe  experienced,  capable  technician*  carrying  out 
the  operation.  They  also  have  the  opportunity  to  practice  themselves 
during  this  period.  The  third  and  final  phase  of  the  program  is  the 
written  and  practi  al  test  of  the  operator.  Certification  cards  are 
issued  to  those  vlic  pass  the  written  teat  and  successfully  fabricate  the 
teat  samples.  Make-up  lectures,  demons trationa ,  and  testa  are  provided 
for  those  who  silfs  ersslons  for  valid  reaaons.  Also,  retesting  is 
provided  after  s<rr--  additional  Instruction  for  those  who  fail  the  first 
time.  Reassignment  la  reconnended  for  applicants  vho  repeatedly  fall 
the  same  test.  Outline#  of  typical  programs  are  presented  in  Appendixes 
I  and  II. 

After  less  than  a  yrri  of  operation,  the  operator  and  inspector  training 
and  certification  ha*  resulted  In  Improved  workmanship  and  fewer 


Tea ting  Procedures 

Inspecting  Arc  Welding 
Hardness  Testing 
Magnetic  Particle  Testing 
Penetrant  Testing 
Tensile  Testing 
Ultrasonic  Testing 
Interpreting  Radiographs 


rejections,  He  have  sot  run  controlled  experiments  to  obtain  reliability 
data,  bat  there  it  no  doubt  Iron  the  other  evidence  that  potential 
failnraa  from  processing  deficiencies  are  being  eliminated,  and  that  we 
may  eventually  reach  a  manufacturing  process  failure  rate  approaching 
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APPENDIX  I 


SMUT  PAINTING 


A.  Lecture  (2  hours) 

1.  Principles  of  Spraying  -  reason;  types  of  spray. 

2.  Equipment  and  Facllifclea  -  guns;  systems. 

3.  Preparation  of  Coating  Materials  -  thinner*; 
primers;  storage,  etc. 

4.  Procedure  and  Operation  -  preparation  of  surfaces; 
technique;  touch-up;  care  of  equipment, 

5.  Inspection  of  Coatings  -  continuity,  thickness; 
adhesion. 

3.  Qualification  Requirements  (2  hours) 

1.  Written  Questions  on  Lecture 

2.  Practical  Exercise  -  spraying  of  three  (3)  panels, 

C.  Requalification  (2  hours) 

1.  Annual  practical  Exercise  -  spraying  of  three  (3) 
p  mel  * . 
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APPENDIX  II 


INSPECTION  OF  MANUAL  ARC  WELDING 


A.  Lecture  (1%  hour*) 

1.  Requirement*  of  Welding  Inspector 

2.  Duties  of  Welding  Inspector: 

Interprets tlon  of  drawings  end  specifications;  verifi¬ 
cation  of  procedure;  selection  of  test  samples;  inter¬ 
pretation  of  test  results. 

3.  Weldment  Defects 

4.  Testing  of  Welds  and  Welded  Joints: 

Chemical;  metallographic;  mechanical;  visual;  mag 
netlc  particle;  penetrant;  radiographic;  ultrasonic. 


B.  Demonstration  and  Application  (1  hour) 


C.  Qualification  Requirements  (1  hour) 

1.  Written  Examination  on  Lecture 

2.  Macroscopic  Examination  of  Weldment 

3.  Identification  of  Five  Defects  in  Weldment 


D.  Requallflcatlon  (3  hour) 

1.  Annual  Practical  Exercise  -  Macroscopic  examination 
of  weldment;  Identification  of  three  defects  in  weld¬ 
ment. 


1  -  Ub9 


I 


HUMAN  FACTORS 
AND 

MISSILE  SYSTEM  UNRELIABILITY1 

Charles  Bates,  Jr.,  Aerospace  Medical  Laboratory 
Wright  Air  Develojment  Division 

John  E.  Short,  Directorate  of  Systems  Management 
Wright  Air  Development  Division 

Albert  Shapero,  Stanford  Research  Institute 
Menlo  Park,  California 


INTRODUCTION i 

Anyone  associated  with  the  test  of  missile  systems  has  exper¬ 
ienced  cleaning  rags  left  in  LQX  lines,  inaccurately  read  meters, 
and  switches  inappropriately  activated.  Unfortunately,  these  ‘mal¬ 
functions'  always  seem  to  occur  at  especially  critical  times  in  the 
test  program  and  usually  go  undetected  until  the  investigation  of 
the  test  failure.  When  these  human  errors  are  exposed  by.  investiga¬ 
tion,  there  is  usually  very  little  constructive  action  taken  to  pre¬ 
vent  their  reoccurrence.  Generally  this  kind  of  system  malfunction 
is  categorized  as  ‘human  error*  or  ‘random  failure*  and  dismissed 
with  the  assumption  that  as  long  as  you  have  people  in  a  system, 
such  things  will  occur. 

Because  of  the  lack  of  concrete  evidence  on  the  magnitude  of 
human  error  in  Air  Force  missile  test  prograns,  the  Aerospace  Medi¬ 
cal  Laboratory,  WADD,  in  cooperation  with  the  Directorate  of  Systems 
Management,  VADD,  instituted  a  research  progran  to  attempt  to  estab¬ 
lish  the  scope  of  this  problem,  and  if  poasible,  to  propose  some 
practical  solutions.  It  is  the  result  of  this  research  effort  that 
we  wish  to  discuss  here.^ 


t  1.  The  research  reported  here  was  performed  under  Contract 

AF  33(6l6)-5666  with  the  Stanford  Research  Institute,  Menlo  Park, 
California,  and  is  more  fully  reported  in  WADD  TR  60-3&. 

2.  The  authors  wish  tc  express  their  appreciation  to  Brig. 
Gen.  Don  Flick:  ger  for  his  encouragement  during  the  initiation  of 
this  research  effort. 
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RESEARCH  PROGRAM; 


The  urgency  of  this  problem  dictated  that  several  limitations  be 
placed  on  the  research  effort  from  the  outset.  First  It  was  deter- 
'r-'-ed  not  to  collect  or  generate  any  new  raw  data.  It  was  believed 
that  more  than  enough  data  existed  in  the  form  of  range  records,  mal¬ 
function  reports,  countdown  recordings,  and  unscheduled  hold  reports 
to  provide  reliable  evidence  for  the  scope  of  the  problem  being  inves¬ 
tigated.  Secondly,  due  to  time  limitations,  it  was  believed  necessary 
to  limit  our  sample  of  missile  systems  whose  test  programs  would  be 
investigated,  and  to  consider  only  Air  Force  missile  systems. 


During  our  preliminary  contacts  with  Air  Force  agencies  and  con¬ 
tractors  to  make  arrangements  for  obtaining  the  necessary  test  data, 
extensive  interviews  were  obtained  with  as  many  contractor  end  mili¬ 
tary  groups  concerned  with  the  development  of  missile  systems  as  were 
believed  to  possess  experience  with  human  error  in  test  progrons. 
These  interviews  and  all  subaequent  data  were  obtained  from  personnel 
connected  with  the  following  missile  systems  and  contractors! 


Missile  System 

Atlas 
Bom  arc 
Goose 
Hound  Dog 

Mace 

Matador 

Titan 

Nose  Cone  System  for  Atlas 
and  Thor 

Snark 


Contractor 

Convair,  San  Diego 
Boeing  Airplane  Co. 
Fairchild  Airplane  Co. 
North  American  Aviation, 
Inc. 

Martin  Co. 

Martin  Co. 

Martin  Co. 

General  Electric 
Northrop  Corp. 


Our  sample  of  systems  therefore  included  four  strategic  surface- 
to-surface  missiles,  two  tactical  surface-to-surface  systems,  one 
surface-to-air,  and  one  air-to-surf ace  missile  system.  One  major 
missile  subsystem  was  eJ-so  included.  As  was  pointed  out  previously, 
because  of  time  limitations,  only  Air  Force  missive  systems  were  in¬ 
cluded  in  this  study. ^ 


1.  The  authors  wish  to  express  their  appreciation  to  the  many 
Air  Force  and  contractor  personnel  who,  through  their  cooperation, 
greatly  facilitated  the  completion  of  this  project.  In  many  in¬ 
stances,  entire  case  histories  of  missile  launch  activities  were  made 
available  for  analysis. 
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As  a n  initial  step  in  this  research  effort,  a  review  was  con¬ 
ducted  of  weapon  system  documentation  in  an  attempt  to  determine  if 
any  formal  attempt  was  being  made  to  cope  with  the  human-initiated 
malfunction.  The  reports  reviewed  included  those  dealing  with  pro¬ 
gram  management,  test  planning,  testing,  malfunction  reporting, 
reliability,  and  human  engineering,  from  the  review,  it  was  found 
that  t 


1.  Present  malfunction  reporting  systems  are  inadequate  in 
Identifying  human-initiated  error  or  for  providing  analyzable  data 
pertinent  to  human-initiated  malfunctions  that  could  be  used  in  orig¬ 
inating  appropriate  corrective  action  or  for  cross-comparing  human- 
initiated  malfunction  among  different  missile  systems.  Though  the 
portion  of  the  equipment  failure  reports  and  unscheduled  missile  holds 
reviewed  that  displayed  strong  evidence  of  human  error  was  large, 
these  malfunctions  were  not  reported  in  adequate  form. 

2.  Little,  if  any,  formal  or  systematic  him  an  engineering 
testing  is  being  conducted.  Of  the  nine  prograna  reviewed,  only  one 
indicated  any  evidence  of  attempting  a  human  engineering  performance 
test  program.  This  one  approach,  which  was  only  a  plan  at  time  of 
review,  was  briefly  described  as  a  program  far  observing  blockhouse 
operations  in  order  to  determine  their  adequacy,  efficiency,  and 
safety.  No  further  detail  was  available  at  the  time  of  review. 

Since  no  evidence  was  found  of  past  or  currently  active  human 
factors  engineering  performance  testing,  the  subsequent  review  of 
the  data  was  primarily  devoted  to  a  determination  of  the  frequency  of 
human-initiated  malfunction  and  an  analysis  of  the  current  practices 
employed  in  reporting  these  data. 

FREQUENCY  OF  HUMAN -INITIATED  MALFUNCTION » 

An  effort  was  mode  to  ascertain  the  relative  frequency  of  bumaa- 
initiated  malfunction  from  available  malfunction  reports  from  Sevan 
of  the  nine  missile  systems  listed  above.  In  addition  to  these  data, 
estimates  of  human-initiated  failures  were  obtained  from  human  engi¬ 
neering  personnel  who  had  independently  conducted  such  analyses. 

Since  the  philosophies  and  methods  of  identifying  and  collecting  mal¬ 
function  data  vary  considerably  among  contractors,  and  even  between 
divisions  of  the  same  contractor,  and  since  the  types  of  missile,  the 
periods  covered  by  the  data,  tbo  number  of  missiles  represented,  and 
the  extent  of  system  development  were  different  for  each  missile  sys¬ 
tem  examined ,  the  data  collected  on  any  one  miasile  system  are  not 
comparable  with  that  collected  on  any  other  system-  The  results  for 
any  single  system  have  been  obtained  by  using  the  data  available  on 
that  system  in  its  own  terms.  Because  of  the  limitations  imposed  by 
the  data,  the  results  here  reported  must  be  considered,  therefore,  to 
represent  an  attempt  to  determine  conservatively  the  relative  impor¬ 
tance  of 'human-initiated  malfunction  in  missile  system  test  prograna. 
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Each  of  the  malfunction  reports  collected  was  analyzed  and  the 
malfunction  was  classified  as  either  human-initiated  or  equipment- 
initiated*  in  equipment  failure  or  an  unscheduled  hold  was  considered 
to  be  human-initiated  if  the  human  component  could  be  clearly  identi¬ 
fied  as  the  cans  at  ire  agent  in  the  immediate  train  of  events  leading 
to  the  system  malfunction.  Equipment  failures  that  could  be  ascribed 
to  such  causes  as  misasaanbly,  mishandling,  or  Disadjustment  by  ths 
operator  were  identified  as  human-initiated.  Failures  that  ultimately 
m.'ght  be  ascribed  to  auch  causes  as  poor  design  were  not  so  identified. 

The  specific  procedure  followed  in  identifying  humen-J nl tinted 
failures  varied  according  to  the  manner  in  iftiich  the  original  failure 
data  were  recorded.  In  most  instances,  the  failure  report  system  em¬ 
ployed  extensive  failure  codes  vfcich  permitted  gross  sorting  by  code. 

In  systems  lacking  some  coding  scheme,  individual  failures  were  re¬ 
viewed  individually  with  cognizant  personnel  from  the  miasile  con¬ 
tractor  organization.  Vhere  failure  codes  were  employed,  the  following 
steps  were  taken t 

1.  The  reports  were  sorted  to  separate  from  the  total  thoae 
equipment  failures  that  were  clearly  not  human-initiated.  Examples  of 
failures  thus  classified  were  those  coded  as  •microphonia* ,  "fungus 
effect*,  ‘hysteresis*,  'loss  of  residual  magnetism*,  etc. 

2.  The  remaining  reports  were  sorted  to  separate  thoae 
equipment  failures  that  were  clearly  human-initiated.  Examples  of 
failures  thus  classified  were  those  coded  as  'human  error*,  'reversed 
leads* ,  ‘wrong  part  installed*,  ‘torque  incorrect* ,  etc. 

3.  The  remaining  reports  were  examined  in  further  detail 
before  they  were  classified.  For  instance,  where  the  failure  was 
coded  as  ‘broken  wire*,  it  was  not  classified  as  human -initiated  unlesB 
there  was  supporting  evidence  indicating,  for  example,  that  it  was 
broken  by  being  placed  under  too  much  tension  during  a  maintenance 
operation  performed  by  an  individual. 

The  data  collected  consisted  of  3*829  equipment  failure  reports 
and  419  unscheduled  hold  reports.  The  largest  number  of  failure  or 
hold  reports  for  any  one  missile  was  1,391.  the  smallest  number,  130. 

Of  the  3.829  failure  reports  analyzed,  29  percent  were  classified  as 
human-initiated  failure,  and  of  the  419  hold  records,  20  percent  were 
classified  as  human-initiated.  For  individual  missile  systems, 
human- ini tiated  error  ranged  between  20  and  53^»  es  follows  1 
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Mi  as lie 

A 

B 

C 

0 

X 

F 

G 

$ 


%  Total  Failure  Reports  Examined 

23 

20 

26 

42 

26 

53 

52 

36 

36 


Because  of  the  limitations  of  the  available  datat  no  precise  as¬ 
sessment  can  be  made  of  the  relative  frequency  of  mlaalle  system  mal¬ 
functions  attributable  to  human  initiation*  The  data  are  sufficient, 
nowever,  to  warrant  the  conclusion  that  a  large  portion  (l/5  to  1/2) 
of  all  missile  system  malfunctions,  perhaps  the  largest  single  iden¬ 
tifiable  portion,  is  human-initiated.  This  conclusion  is  further 
supported  by  the  following: 
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1.  There  is  a  good  possibility  that  many  of  the  equipment 
failures  reviewed  in  the  reports  that  were  classified  as  not  human- 
initiated  were  human-initiated.  For  example,  a  failure  coded  only  as 
•overloaded*  may  have  been  initiated  by  sn  operator  connecting  28 
volt  equipment  to  a  110  volt  source.  In  the  absence  of  additional  in¬ 
formation,  this  failure  would  not  have  been  classified  as  human- 
ini  tinted. 

*.  There  is  evidence  to  support  the  conclusion  that  many 
human-initiated  malfunctions  are  not  reported  in  order  to  avoid  any 
implication  of  fault  cn  tne  part  of  reporting  personnel.  During  the 
course  of  this  study,  interviews  with  contractor  test  personnel  re¬ 
vealed  that  at  least  one  disastrous  launch  or  flight  failure  in  each 
of  the  programs  under  review  was  human-initiated.  This  fact  was  sel¬ 
dom  reflected  in  malfunction  reports  or  flight  failure  reports.  For 
exauple,  in  one  Frd)  program,  several  successful  launches  had  beet,  ex¬ 
perienced  using  the  snr.e  crew  and  equipment  until  one  occasion  when  a 
technician  connected  a  plug  into  the  wrong  one.  of  two  adjacent,  iden¬ 
tical  receptacles.  This  error  resulted  in  a  scrubbed  launch  and  a 
damaged  missile.  An  examination  of  the  reports  concerning  this  major 
failure  revealed  no  indication  cf  this  sequence  of  events.  Further¬ 
more,  interviews  with  reliability  and  design  personnel  indicated  that 
even  informal  iu.owle.gc  i.  f  t:,is  occurrence  had' not  been  made  available 
to  then. 


".1 


'  •■'••'.'■'.'s'  -/ 

w* *•  —  a.ji, 


1.  Identity  of  the  systems  has  been  intentionally  obscured. 

2.  Conp.u.y  estimates  based  cn  independent  studies. 
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CURKEKT  RiUCTIGlo  IR  REFORTIi'G  KUMAN- INITIATED  MALFUNCTIONS i 

As  Doted  above,  most  malfunction  reporting  schemes  employed  some 
method  of  coding  malfunctions.  Several  of  the  systasa  reviewed  em¬ 
ployed  some  explicit  code  term  for  human-initiated  failure.  The  re¬ 
sults  of  this  study  indicate  that  perhaps  the  use  of  such  code  terms 
as  "human  error"  tend  to  diminish  the  number  and  honesty  of  failure 
reports. 

Sven  in  cases  where  the  procedures  or  equipment  were  inadequately 
designed,  thereby  farcing  the  operator  to  initiate  the  causal  chain 
leading  to  a  failure,  it  is  highly  probable  that  the  operator  does  not 
choose  the  term  "human  error"  because  of  its  implications  of  guilt. 

The  avoidance  of  this  term  is  borne  out  by  the  results  of  the  analysis 
of  the  failure  data  employing  "human  error"  as  a  code  term.  Both  of 
these  systems  rely  on  the  operator  to  report  failure  data.  The  re¬ 
sults  were  as  follows i 


Malfunctions 

Malfunctions 
Analyzed  as 

Missile 

U  Malfunctions 

Coded  as 

Human-Initiated 

System 

Reported 

Human  Error 

in  this  Protect 

A 

1.391 

3 

322 

B 

977 

0 

193 

Little  evidence  was  found  in  the  various  malfunction  report  forms, 
instructions,  and  procedures  of  any  attempt  to  alert  the  reporter  or 
analyst  to  a  consideration  of  the  possible  effect  of  the  human  compon¬ 
ent  in  the  immediate  causal  chain  leading  to  failure.  In  systems 
where  the  term  "human  error"  was  used,  it  appears  that  the  existence  of 
the  term  itself  was  considered  sufficient.  As  for  mechanism  to  assure 
corrective  action,  only  one  of  the  reviewed  systems  included  a  state¬ 
ment  to  the  effect  that  failures  initiated  by  humans  should  be  referred 
to  human  engineering  personnel. 


Without  exception,  the  failure  reporting  systems  reviewed  were 
designed  to  identify  and  report  on  specific  equijment  components.  In 
some  ceses,  the  malfunction  report  had  to  be  accompanied  by  the  failed 
part.  Since  human-initiated  malfunctions  cannot  in  present  reporting 
systems  be  adequately  described  in  terms  of  a  discrete.  Identifiable 
entity  such  as  the  equipment  component,  only  the  narrative  statements 
on  the  reporting  forms  can  be  used  for  their  identification.  The  use 
of  narrative  data  m  malfunction  reports  was  found  to  be  unr at is factory 
fcr  this  purpose  in  many  respects.  It  was  pointed  out  many  times  by 
missile  contractor  porscanel  being  interviewed  that  it  was  difficult  to 
procure  such  data  from  o;-erating  personnel.  In  addition,  the  physical 
space  that  is  provided  fcr  the  narrative  statement  on  most  failure 
forms  is  so  limited  as  to  discourage  a  full  description  of  the  events 
preceding  the  failure. 


One  additional  serious  disadvantage  of  equipment  malfunction  forma 
for  the  reporting  of  human-initiated  malfunction  data  is  their  failure 
to  report  malfunctions  that  do  not  result  in  equipment  failure,  ouch 
procedural  malfunctions  ara  ns  capable  of  falling  a  mission  as  hard- 
uqr®  failure  and  are  as  indicative  of  ayatem  unreliability  and  required 
redesign  of  the  system  or  its  equipment  as  are  purely  hardware  mal- 
func tions. 

PROPOSED  SOLUTION! 

The  investigatora  in  this  project  wre  impressed  with  the  positive 
contribution  made  to  reliability  by  the  human  component  when  effectively 
utilized.  In  the  majority  of  the  human  errors  that  could  be  isolated 
and  analyzed,  human-initiated  failure  was  attributable  to  the  improper 
use  of  the  human  as  a  component  in  the  system  or  the  violation  of  the 
most  elementary  principles  of  human  engineering  in  the  deeign  of  the 
system  hardware.  Such  items  as  mistaking  red  power-on  lights  for  mal¬ 
function  indications,  switching  through  missile  power  to  shutdown,  and 
four  operators  attempting  to  co.anunicate  simultaneously  on  the  sane 
hot  line  are  examples  of  the  human  errors  discovered.  Basea  on  suoh 
evidence,  the  authors  firmly  believe  that  intensive  human  engineering 
of  both  the  system  and  its  hardware  is  necessarily  the  basis  of  any 
solution  to  the  human  error  problem.  In  essence,  this  would  amount  to 
directing  the  same  mount  of  human  engineering  attention  to  our  missile 
test  programs  as  we  devote  to  the  design  of  our  operational  systems. 

In  the  majority  of  the  systems  surveyed,  the  contrast  between  missile 
flight  test  equipment  and  the  functionally  3ame  equipment  designed  for 
operational  use  is  dramatic.  Alert  management  on  several  of  the  sys¬ 
tems  surveyed  have  recognized  this  problem  and  have  instituted  the  pol¬ 
icy  of  designing  equipment  to  operational  configuration  as  nearly  as 
possible  from  the  beginning  of  system  design  and  utilizing  this  equip¬ 
ment  in  the  missile  flight  test  program.  Such  an  approach  has  many 
advantages,  not  the  least  of  which  is  the  ability  to  ‘flight  test* 
more  than  the  air  vehicle  alone. 

Coupled  with  the  increased  participation  of  human  engineering  per¬ 
sonnel  in  the  design  of  test  systems,  we  propose  that  the  human 
engineer  participate  in  system  design  in  a  fashion  more  in  keeping  with 
other  engineering  disciplines.  In  essence,  the  design  of  a  system  or 
its  components  is  an  iterative  process  that  includes  such  elements  as 
system  analysis,  uesign  of  the  hardware,  performance  prediction,  test 
of  the  design,  and  either  design  acceptance  or  modification  based  on 
test  results.  The  profession  of  human  engineering  has  traditionally 
participated  in  system  analysis  and  hardware  design,  but,  as  the  evi¬ 
dence  from  this  study  indicates,  little  effort  has  been  devoted  to 
human  engineering  testing.  For  such  a  human  engineering  te3t  program 
to  be  practical,  it  is  necessary  to  consider  the  selection  of  entitles 
appropriate  tc  the  human  component  which  can  provide  the  basis  for 
design,  performance  ireaiction,  test,  data  collection,  and  system  mod¬ 
ifications  as  they  concern  the  system  human  components.  These  entities 
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;nust  be  capable  of  being  referred  to  existing  or  potential  models  that 
describe  the  dynamic  relationships  of  the  human  component  ■within  a 
weapon  system.  To  be  considered  appropriate,  these  entities  must  meet 
the  following  criteriai 

1.  They  must  be  readily  and  consistently  definable,  observ¬ 
able,  and  measurable  by  those  concerned  with  the  system  test. 

2.  They  must  be  referable  to  existing  models  of  the  human 
components  of  a  weapon  system. 

3.  They  must  be  subject  to  design  and  modification. 

From  tne  viewpoint  that  man  participates  in  a  -weapon  system 
through  the  operations  he  performs,  it  appears  appropriate  to  use  the 
operation  os  the  entity  that  is  the  human  engineering  equivalent  of 
the  hardware  designer's  "black  box*.  An  operation  could  be  defined  as 
any  process  that  translates  a  system,  or  a  portion  of  a  system,  from 
State  A  to  State  B.  With  the  humun  operation  concept,  the  human  engi¬ 
neering  test  program  as  proposed  here  would  consist  of  the  following 
steps. 

1.  Identification  and  selection  of  critical  human  operations. 

2.  Specification  of  pertinent  parsneters  of  these  operations. 

3.  Prediction  cf  the  values  of  these  parameters. 

Confiraation  through  test  of  predicted  values. 

5.  Acceptance  or  redesign. 

Although  the  human  c-eraticn  is  proposed  as  the  ha-nan  engineering 
black  box,  only  "critical  operations*  in  a  system  would  be  subjected 
to  test.  These  critical  operations  are  defined  as  these  human  opera¬ 
tions  which,  if  not  perfom-od  in  accordance  with  estimated  design 
values,  would  have  severe  effects  on  system  operation  or  cost.  For 
example,  any  cjiVutita  which,  if  improperly  performed,  might  cause  a 
hold  in  a  cuuntJc-r, ,  can  be  censidere-  a  critical  operation.  A  second 
type  of  critical  wperut.„n  would  be  one  which,  if  improperly  performed, 
would  lead  to  increased  costs,  as  for  instance,  an  im.roperly  per¬ 
formed  checkout  routine  wtucn  w  uld  lend  to  the  rejection  of  a  "good" 
missile.  An  oj-er-itioi,  m.iy  be  defined  as  critical  if  it  falls  into  one 
or  .-no re  cf  the-  fell  vis,'  categories  or  combinations  of  the  s  categories  1 

1.  Tine -V,_iued  C»  er.itions  -  operations  which,  if  mal per¬ 
formed  ,  ;:.ay  cause  time  Jo  lay 3  in  the  operating  or  support  cycles  of 
the  weapon  system. 

2.  Accuracy-Valued  C;  er.--.tu ns  -  operations  which,  if 
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raalper formed,  may,  for  exsnple,  cause  the  missile  to  miss  the  target 
or  the  warhead  to  detonate  improperly. 

3.  Coat-Valued  Operations  -  operations  which,  if  performed 
i.n,. cperly,  nay  increase  the  coat  of  the  operation. 

Cnee  the  critical  operations  have  been  Identified,  the  human 
engineering  test  program  would  be  r, jnducted  much  a a  other  engineering 
test  programs.  The  iterative  processes  of  parameter  prediction, 
design,  test,  evaluation  of  test  results,  and  acceptance  or  rejection 
of  the  design  would  be  performed  on  the  critical  operations  seleoted 
for  test,  With  such  a  program,  only  one  major  difficulty  remains, 
that  of  collecting  usable  human-initiated  malfunction  or  failure  data. 
3uch  data  is  necessary  for  the  test  and  evaluation  of  both  critical 
and  con-critical  operations  as  well  es  serving  as  an  important  check 
on  the  identification  of  the  critical  operations  to  be  tested.  It  le 
proposed  here  that  a  relatively  simple  modification  be  accomplished 
on  the  present  malfunction  reporting  forms.  Once  the  test  mission  is 
analyzed  into  its  human  operations,  it  would  only  be  necessary  to 
make  it  possible  for  the  operator  to  identify  on  the  failure  report 
forte  in  which  operation  a  failure  occurred  to  make  the  data  useful  for 
human  engineering  analysis.  The  identification  of  the  operation  would 
make  it  possible  to  reference  the  failure  event  to  those  models  of  the 
system  that  do  describe  the  dynamic  interactions  of  the  failed  item 
with  the  human  component.  With  a  model  such  as  task  analysis,  it  would 
be  possible  to  reconstruct  the  mam-machine  relationships  that  are  in¬ 
volved  to  determine  the  demands  made  upon  the  operator  and  to  identify 
the  important  environmental  conditions.  This  process  would  be  analog- 
C-j  to  referring  a  failed  component  to  the  appropriate  circuit  diagram. 

This  relatively  simple  extension  of  present  reporting  systems 
would  inevitably  lead  tc  improvemen t  in  understanding  exactly  in  which 
areas  and  in  which  ways  the  human  contributes  to  failure,  holds,  and 
other  malfunctions  and  how  human  operation  can  be  modified  to  help 
achieve  desired  missile  checkout,  launch,  and  flight  performance.  Con¬ 
ceivably  the  data  then  received  could  also  be  used  in  the  analysis  of 
the  human  contribution  to  missile  system  reliability.  At  the  very 
least  tney  would  supplement  and  possibly  replace  much  of  the  data 
fathering  techniques  presently  being  employed  that  have  not  proven  to 
be  adequate.  Techniques  3uch  u3  interview  and  observation  could  still 
be  utilized  but  t;.ey  would  no  longer  oe  the  primary  date  gathering 
jpe  thods . 


SUdduAHY : 

The  data  in  the  pxesent  study  indicate  conclusively  that  the  con¬ 
tribution  of  human-initiated  .malfunctions  to  system  unreliability  '9 
of  a  significant  magnitude.  It  appears  thut  even  a  partial  solution 
to  this  i-roclesc  cculu  contribute  substantially  to  the  attainment  of 
the  high  reliability  necessary  for  our  forthcoming  missile  and  space 
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systems.  The  program  outlined  above  of  human  engineering  performance 
testing  of  critical  system  operations  and  improved  collection  of  mal¬ 
function  data  is  believed  to  be  the  most  fruitful  first  approach  to 
thia  problem.  What  is  most  nseded  at  this  time  is  a  systematic  trial 
of  thia  testing  concept  to  insure  its  practicality  and  to  develop 
ertcund  rules  for  its  application.  Plans  are  now  being  formulated 
within  the  W ADD  weapon  system  development  complex  to  subject  the  above 
proposed  program  to  a  test  during  the  research  and  development  phase 
of  s  weapon  system.  It  Is  anticipated  that  such  a  trial  will  result 
in  an  approach  to  the  system  reliability  problem  that  would  merit 
serious  consideration  for  general  application. 
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PERSONNEL  LOGISTICS  AS  A  FACTOR  IN  ACHIEVING 
WEAPON  SYSTEM  RELIABILITY 


D.  Meister 

Convai r -Astronautics 

Engineers  have  been  concerned  about  the  reliability  of  their 
equipments  for  a  long  time,  eveu  long  before  thia  concern  «aa 
channeled  into  what  today  la  called  reliability.  Originally  thia 
concern  waa  for  the  performance  without  failure  of  Individual  ma¬ 
chine*  or  groapa  of  aachinea.  Recently,  however,  the  concept  of 
the  "syste*"and  the  need  for  "system  reliability*  haa  becoae 
doainant. 

Thia  *ayatea”  now  encoapaaaea  not  only  groupa  of  aachinea  per¬ 
forming  a  unified  operational  fnnction  but  alao  the  personnel  who 
operate  and  aaintain  theae  aachinea.  The  syatea  functions  not 
only  deal  with  the  ultimate  tactical  operation**  in  guided  aiasiles, 
the  launch-  but  also  with  all  of  its  associated  supportive  oper¬ 
ations:  checkout,  preventive  maintenance,  logistics,  planning,  etc., 
all  of  which  involve  varied  personnel  functions  occurring  hours  or 
even  days  prior  to  the  launch,  and,  in  the  case  of  a  deterrent 
weapon  system,  even  if  never  launched. 


To  talk  about  aystea  reliability,  then,  neans  that  one  auat  not 
only  be  concerned  with  the  evaluation  of  equipaent  efficiency,  but 
also  with  the  efficiency  of  the  men  who  operate  and  aaintain  that 
equi paen*  • 

For  this  reason  Convair  reliability  engineers  have  for  soae  years 
been  concerned  about  the  effect  of  "human  factors"  on  reliability 
and  rightly  so.  Our  experience  at  Convai r-Astrenauti cs  suggests 
that  when  an  equipment  aystea,  or  any  part  of  it,  makes  the  tran¬ 
sition  from  laboratory  testing  to  testing  by  field  personnel;  or 
after  factory  testing,  is  handed  over  to  military  personnel  for 
operational  use,  a  considerable  drop  in  reliability  (system  per¬ 
formance)  occurs.  If  we  could  measure  these  things  in  a  sophis¬ 
ticated  fashion,  we  would  probably  be  able  to  confirm  this  quanti¬ 
tatively. 

Various  explanations  are  given  for  thia  loss  of  system  relia¬ 
bility.  When  reliability  drops  after  we  transfer  the  system  to 
military  control,  we  say  that  military  personnel  are  not  in¬ 
itially  as  adequately  trained  aa  our  civilian  technicians;  hence 
the  former  encounter  greater  difficulties.  But  something 
similar  occurs  when  our  own  field  test  personnel  receive  equip¬ 
aent  for  testing.  Convai r-Aatronauti cs  Operational  Roporta, 
which  are  sent  into  the  plant  by  our  field  sites  when  oper¬ 
ational  problems  arise,  describe  many  instaoees  In  which  system 
performance  is  inept.  Why  °  What  are  the  factors  that  result 
in  reduction  of  reliability  after  equipment  is  put  into  the 
hands  of  any  new  group  of  using  personnel? 
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The  first  factor  is  the  sat  engineers  are  Boat  familiar  with. 

This  i  a  design  weakness  which  prototype  and  preproduction  test¬ 
ing  have  not  revealed,  but  wbieh  the  presumably  more  stressful 
operational  environment  (whether  test  site  or  operational 
aits)  reveals  for  the  first  time.  The  operation  of  multiple  sub- 
-y. *»ms  together  for  the  first  time  may  introduce  an  incompati¬ 
bility  resulting  in  failure.  This  is  very  likely  to  be  the  ease 
where  systems  are  designed  and  tested  as  individual  packages  by 
different  associate  contractor^..  The  result  of  a  design  de¬ 
ficiency  is  usually  a  functional  failure  of  the  equipment;  that 
is.  the  equipment  simply  fails  to  operate  at  all.  or  operates  in 
so  erratic  a  fashion  that  it  rapidly  ceases  to  function.  Such  a 
design  deficiency  in  almost  all  eases  winds  up  as  an  equipment 
failure  report, generated  by  using  test  personnel,  in  the  hands 
of  the  reliability  engineer. 


The  second  factor  ie  one  which  engineer" are  beginning  to  re¬ 
cognise  more  and  more:  inadequate  design  of  machine!  for  human 
operation  and  maintenance.  By  this  la  meant  auch  things  aa 
Improper  layout  of  control  panela,  i nacceaai bi 1 i ty  of  components 
for  maintenance,  failure  to  provide  sufficient  controle-  all 
hoee  factors  that  mska  the  operation  and  maintenance  of  a  piece 
jf  equipment  by  a  man  difficult,  tsdious  or  highly  subject  to 
error.  Thla  type  of  inadequacy  ie  not  usually  recognized  at  the 
prototype  and  preproduction  testing  stage,  because  machines  at 
thie  stage  are  net  operated  or  maintained  in  anything  comparable 
to  operational  conditions,  uaing  set  procedures  performed  In  a 
required  time  period.  Operation  and  maintenance  at  tho  proto¬ 
type  tent  stage  resembles  troubleshooting  more  than  operational 
use.  There  la  then  little  opportunity  to  discover  inadequate 
design  fsr  human  uss  unless  specif io  atndiea  of  man-machine 
operation  that  similate  operational  conditions  are  made  at  thla 
time  by  a  team  of  design,  test  and  reliability  human  engineers. 

The  result  of  poor  human  engineering  design  ie  not  generally 
functional  failure  of  the  equipment,  el  though  it  may  preduo#  an 
error  that  does  lead  to  rob  a  failure.  In  moat  cases  the 
equipment  can  he  operated  and  maintained,  but  with  difficulty. 
This  difficulty  Is  manifested  in  increased  time  to  perform  re¬ 
quired  procedures  on  the  equipment;  and  increased  down  time 
when  adjustment  and  maintenance  must  h"  performed.  Bine#  the 
usual  loss  is  time,  this  is  almost  never  reported  ss  a  defect 
against  the  equipment.  Kstimstes  of  lost  time  can  bs,  however, 
sgtrsmely  useful  In  estimating  system  reliability.  At  Convair- 
AsUonsulIra  our  Operational  ((sport  rsqulrse  that  an  eatimats 
of  Mis  t i ms  Inst  from  accomplishing  a  testing  test  reported, 

The  third  factor  Is  one  that  has  received  little  o>  at- 

tentlun  to  dale,  hut  which  cun  seriously  reduce  ayilem  relia¬ 
bility  sveii  th  nugh  equipment  has  been  designed  correct)  for 
f  ii  n  r  1 1  ii  n  a  I  use  and  for  use  by  personnel.  Tills  third  factor 
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involves  a  class  of  husian  factors  problems  which  nay  be  called 
"personnel  logistics".  Personnel  logistics  (or  PL,  for  short) 
deals  with  supportive  aids  for  the  nan  in  the  system  just  as 
equipment  logistics  deals  with  support  for  the  machine.  Table  1 
'  •’dicates  what  PL  encompasses: 

Table  1 

Personnel  Logistics  Means  that 

1.  Written  and  Unwritten  Procedures  and 
Job  Inf ormation ; 

2.  Tools,  Teat  Equipment  and  Spares; 

3.  Required  Transportation; 

4.  Training  and  Training  Aids; 

5.  Physical  Work  Environment 

Are  Optimal  and  Available  When  Needed. 

To  be  entirely  correct  and  additional  PL  factor  should  be  added, 
morals-  This  covers  such  items  as  opportunity,  recognition, 
organization  and  leadership.  Bnt  moat  systems  (and  the  organ¬ 
izations  that  develop  systems)  either  ignore  this  factor  or 
consider  it  as  one  which  they  will  deal  with  in  their  own  way. 

How  does  inadequate  PL  affect  a  system's  reliability?  All 
systems-  particularly  guided  missile  weapon  systems-  have  in 
addition  to  their  stated  functions  (or  as  part  of  these  func¬ 
tions)  a  designed  or  specified  reaction  time.  For  example,  if 
a  potential  enemy  allows  os  only  30  minutes  before  our  missile 
bases  will  be  destroyed,  something  less  than  30  minutes  becomes 
the  system's  required  reaction  time.  Greater  delay  then  in¬ 
volves  the  risk  of  destruction  of  our  bases  before  our  weapons 
have  been  launched.  This  is  of  course  a  fallur*  e*  the  system 
level  and  amounts  to  zero  reliability.  Even  wh  «  reaction 
time  cannot  be  specified,  it  is  understood  that  t  ery  effort 
will  be  made  to  reduce  it  to  a  minimum.  The  reason  why  reaction 
time  is  so  important  to  us  in  this  country  is  that  our  weapon 
systems  are  essentially  deterrent  instruments.  Our  missiles 
react  in  defense,  they  do  not  act  in  offense;  hence  our  primary 
reaction  time  is  determined  by  our  estimate  of  what  the  po  - 
tential  enemy  allows  us.  Anything  that  increases  the  system 
reaction  time  beyond  that  specified  reduces  our  system's  relia¬ 
bility  because  it  reduces  the  length  of  time  we  have  to  utilize 
the  weapon.  Poor  system  rel l ability  can  then  be  defined  at 
least  partially  In  terms  of  slower  reaction  time.  Inadequate 
PI,-  such  h*  the  lack  of  proper  procedures-  can  slow  down  a 
system's  response  to  the  point  Mint  its  reliability  is  gravely 
1  mpal  red,  How'’ 

Let  us  ns-turne  Unit,  we  arc  in  i  •  midst  of  ari  operational  Atlas 
countdown  and  a  in  u  I  '  o  i  •  t  i  mi  has  occurred  which,  after  it  has 
been  dia  I,  ieqm,»e  a  replacement  component.  How  will  the 
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various  PL  factors  offset  the  speed  with  which  that  component  is, 
replaced? 

1.  Inadequate  procedures  or  procedures  that  were  not  clearly 
written,  sight  cause  the  Launch  Control  Officer  to  attempt 
to  secure  a  replacement  component  from  the  wrong  source. 

There  would  be  an  indefinite  amount  of  delay  until  he  had 
contacted  the  Maintenance  Officer  and  made  known  his  needs. 

2.  Inability  to  acquire  transportation  will  delay  delivery  of 
needed  replacement  components.  If  the  spare  is  in  the 
maintenance  a res,  the  length  of  time  needed  to  transport  it 
to  the  launch  pad  becomes  vital,  and  must  be  added  to  the 
system's  reaction  time.  In  addition,  If  the  needed  trana- 
portation  vehicle  is  not  immediately  available  and  must  be 
sought  for,  the  reaction  time  is  increased. 

3.  If  the  spare  part  is  not  available,  or  if  it  requires  a 
special  tool  to  install  it,  and  the  tool  is  not  immediately 
available,  additional  time  is  lost. 

4.  If  our  missile  maintenance  mechanic  has  not  been  properly 
checked  out  in  the  installation  of  the  component;  or  has 
not  practiced  the  installation  a  sufficient  number  of  times, 
then  his  reactions  may  be  slow  and  add.  to  the  system  reac¬ 
tion  time. 

5.  If  his  work  environment  is  such  that  it  is  difficult  for  the 
mechanic  to  get  into  position  to  install  the  component,  so 
that  he  takes  precious  time  setting  himself  up  for  his  task, 
it  vill  take  him  longer  to  Install  the  component. 

#.  And  finally,  if  the  mechanic's  attitude  is  such  that  he  is 
uninterested,  bored,  or  resentful,  »e  can  be  sure  that  the 
installation  will  take  measurably  longer.  The  amount  of 
time  lost  by  the  operation  of  individual  PL  inadequacies 
may  be  slight;  but  added  up,  the  total  becomes  respectable. 
For  example,  in  our  studies  of  malfunction  countdowns  we 
have  had  to  add  automatically  10-15  minutes  to  all  mal¬ 
function  correction  times  just  to  get  the  maintenance  man 
from  the  blockhouke  to  the  launch  pad  proper. 

In  the  countdown  malfunction  situation  I  have  juat  described  in 
which  ri<  was  importnnt,  no  error  was  marts  by  the  maintenance 
personnel  involved.  Human  error  could  have  occurred  under  these 
circumstances,  but  it  did  not,  yet  system  reliability  waa  re¬ 
duced.  The  important  point  to  remember  is  that  even  though  no 
human  error  occurred  amt  no  equipment  malfunctioned,  aystem  re- 
lirbt’ilr  was  reduced  because  system  reaction  time  waa  Increased. 
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It  may  appear  that  theae  PL  factor*  are  Merely  obvj  ous,  garden 
variety  typea  with  which  as  engineers  we  need  not  be  vitally  con¬ 
cerned.  However,  1  would  like  to  advance  the  thesis  that  these 
factors  are  responsible  for  at  least  as  Much  system  unreliability 
as  the  equipment  failures  with  which  we  are  vitally  concerned. 

Is  PL  really  important  In  field  operations? 

A  survey  was  made  approximately  a  year  ago  in  which  Convair  field 
personnel  (technicians,  engineera  and  mechanics)  at  our  three 
field  test  bases  were  asked  to  indicate  what  their  major  operat¬ 
ing  problems  were  at  the  time.(l)  A  substantial  part  of  the 
problems  reported  were  of  the  PL  type. 

The  following  are  some  verbatim  quotes,  merely  to  indicate  the 
flavor  of  the  responses: 

1.  Calibration  curves  not  available  for  installed  transducer; 

2.  Need  table  of  resistance  values  for  maintenance; 

3.  Operating  procedure  inadequate; 

4.  Instruct!  o*j  manual  inadequate, 

5.  Schematics  and  drawings  obsolete; 

8.  Procedure  inadequate  or  misunderstood; 

7.  Inaccessible  for  maintenance; 

8.  Availability  inadequate. 

The  relative  emphasis  on  procedures  is  not  fortuitous.  The  in-  ■ 
adequacy  and  unavailability  of  procedures  (construed  in  the 
larger  sense  of  manual,  schematics,  instruction  material  of  all 
types)  is  by  far  the  outstanding  PL  problem  we  face, 

PL  problems  occur  at  all  stages  of  field  operations-  captive 
tests,  flight  tests,  military  operations,  and  they  probably 
extend  beyond  weapon  systems  into  such  entities  as  large  in¬ 
dustrial  organizations.  They  also  affect  the  reliability  en¬ 
gineer  directly  through  the  collection  of  reliability  inform¬ 
ation.  Like  most  Reliability  organizations,  we  at  Convair- 
Aatronautics  have  had  reason  to  believe  that  our  equipment 
failure  information  reported  from  the  field  is  imcomplete  and 
to  some  degree  erroneous.  A  written  que  stl  onnai  re  and  per¬ 
sonal  interview  survey  was  made  of  our  field  personnel  who 
report  these  failures  to  determine  if  they  bad  any  preblema 
that  might  be  renpons i bl e  for  these  deficiencies.  (2)  We 
found  that  reporting  personnel  were  experiencing  grave  dif¬ 
ficulties  for  thr»e  reasons: 

1.  Their  instructions  for  filling  out  failurs  forms  could  be 
improved  in  clarity  and  content  (PL  factor  1-  procedures) 

2 .  Some  of  the  information  they  needed  to  complete  the  forms 
correctly  was  either  unavailable  or  available  ODly  with 
much  difficulty;  (I’L  factor  1  -  job  Information) 

A.  Krns  a  motivational  standpoint,  tliew*  reporting  personnel 
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did  not  know  the  usee  to  which  their  information  waa  being 
pat.  (PL  factor  x  -  morale).  Thia  tended  to  make  them  care- 
leaa  and  indifferent.  Once  these  problem*  were  established 
as  valid,  they  were  remedied  to  aome  extent,  with  consequent 
improvement  in  the  information  reported. 

Such  personnel  logistics  problems  are  not  solved  through  the 
more  adequate  design  of  equipment,  because  they  have  very  little 
to  do  with  equipment  design.  Rather  these  problems  are  adminis¬ 
trative  and  organisational  in  nature;  and  must  be  solved  by 
management  in  the  planning  stages  of  system  development. 

There  is,  fortunately,  nothing  especially  complicated  about  the 
task  of  anticipating  and  forestalling  PL  problems.  In  the  early 
planning  for  field  test  and  ultimate  operation  of  a  system,  the 
various  jobs  that  mak »  up  the  missile  operation  at  an  R  &  D  and/ 
or  operational  site  vast  be  determined.  The  individual  jobs  must 
then  be  analyzed  in  terms  of  what  they  will  require  of  the  various 
PL  factors.  For  example,  take  the  Job  of  transporting  an  Atlas 
from  its  squadron  maintenance  area  to  the  launch  site.  Obviously, 
this  job  requires  a  missile  handling  trailer;  it  needs  special 
procedures  for  installing  the  missile  on  its  trailer;  and  it  may 
require  aome  special  training  of  the  driver.  As  long  as  we  have 
a  description  of  each  job,  we  can  analize  it  in  a  similar 
fashion.  The  task  of  predetermining  PL  requirements  can  be 
summed  up  in  the  form  of  a  matrix  checklist,  in  which  the  various 
jobs  that  describe  the  system  operation  are  listed  running  hori¬ 
zontally  across  the  checklist;  and  the  special  PL  factors  run 
vertically  down  the  checklist.  A  sample  checklist  might  look 
(very  roughly)like  this: 


Procedures 
Special  tools 
Spares 

Transportati on 
Special  Training 
Work  Environment 


Each  of  the  FI,  factors  aud  each  of  the  jobs  may  bs  broken  down 
into  as  molecular  detail  as  is  found  useful.  For  example,  if 
description*  of  the  various  special  tools  that  may  bs  required 
are  available,  they  can  be  listed  by  name  as  sub-categories 
under  the  heading  of  special  tools.  If  Job  1  is  composed  sf  a 
number  of  individual  iaske,  one  may  wish  to  break  out  each 
task  as  a  subeategory  under  job  1.  This  depends  on  ths  amount 
of  information  available  and  the  usefulness  of  suboategorl si ng 
th i  i  Information  for  our  purposes. 


Table  2 
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To  refer  to  table  2,  if  emphasis  on  procedure*  is  required  in 
job  2  and  job  4,  but  not  in  1 ,  3  and  S  (because,  let  us  sa y  the 
latter  are  cosaon  tasks  with  a  great  deal  of  skill  transfer  fro* 
other  types  of  work),  then  eheekmarks  are  placed  in  the  appropri' 
ate  categories.  The  checklist  simply  rewinds  the  planner  that 
■p^cial  care  scat  be  taken  to  provide  detailed  procedures  for 
jobs  2  and  4.  The  its*  analysis  is  performed  for  the  other 
factors.  The  essential  part  of  this  entire  process  is  the  ana¬ 
lysis  of  the  job;  and  the  asking  of  the  particular  question , what 
special  PL  requirements  does  the  job  have? 

If  human  factors  affect  system  reliability,  there  should  be  some 
quantifiable  measure  of  the  effeet  of  these  factors.  At  the 
present  tine  we  hare  no  method  of  measuring  this  effect-  which 
I  have  arbitrarily  called  by  a  short  handlers,  "huaan  relia¬ 
bility",  although  there  are  significant  differences  between 
equipment  reliability  and  hnman  performance  characteristics. 

Present  trends  suggest  that  in  the  f uture_ reli abi li ty  indices 
will  be  used  more  and  more  as  evaluative  criteria  in  weapon 
system  contract  specifications.  It  is  therefore  highly  desira¬ 
ble  that  we  derive  some  measure  of  the  effect  of  the  huaan 
factor  on  system  reliability,  since  an  equipment  reliability 
index  by  itself  provides  only  half  the  necessary  story  to  the 
weapon  system  buyer.  It  is  necessary  to  supplement  or  integrate 
it  with  a  measure  of  the  human  reliability  onecan  or  haa  a- 
chieved. 

There  is  an  important  additional  reason  for  developing  an  index 
of  human  reliability  which  can  be  included  in  contract  specifi¬ 
cations.  This  is  the  pragmatic  reason  that  if  such  a  measure 
is  included  in  contract  specifications,  management  will  pay 
much  more  attention  to  solving  these  human  factors  problems. 

Reliability  engineers  can  define  equipment  reliability  in  terms 
of  equipment  failure  (mean  time  between  failures).  Tbeir  reli¬ 
ability  data  are  generally  translated  into  predicted  failure 
rates . 

In  thinking  About  human  performance  we  tend  to  think  of  human 
error  as  comparable  to  equipment  failure.  An  error  is  a  fail¬ 
ure  of  the  human  organism  to  perform  as  needed.  This  defi¬ 
nition  hAs  not,  however,  proven  very  useful  to  us  in  our  study 
of  human  reliability  for  two  reasons: 

1.  After  the  initial  period  of  learning  a  task  is  over,  errors 
are  relatively  infrequent; 

2.  People  are  reluctant  to  diocuss  their  o*n  or  othei's  errors 
and  therefore  report  them  rather  infrequently. 


At  Convai r-Astronautics  ws  have  triad  various  methods  of  near 
lax  indices  of  huaan  performance.  The  Failure  and  Conauaption 
fora  used-  BWD0-50-  had  at  ona  tiae  a  code  tor  equipment 
failure  resulting  fraa  huaan  error  but  the  nuaber  of  failures 
reported  aa  due  to  huaan  error  vaa  a  very  saall  percentage  of 
•1*  equipment  failures  reported.  In  an  independent  estimate 
booti  on  an  ezaaination  of  5000  equipaent  failure  reports  re¬ 
ceived  we  eatiaated  that  approzimately  20%  of  equipaent  fail¬ 
ures  were  eaused  by  various  huaan  factors*  (3).  The  aaziaua 
percentage  (see  Figure  1 )  of  huaan  errors  reported  over  an  18 
north  interval  was  0Jt  of  the  total  received. 


"Percentage  of  Huaan  Error  Reports  Received* 


(See  Page  1  -  172  for  Figure  1) 


Figure  1 

Another  fora  used  at  Convai r-Astronauti ca  is  ths  Operational 
Report  (OS)  which  was  referred  to  previously.  Aaong  the  cate¬ 
gories  of  probleaa  requiring  an  OR  is  a  huaan  factors  problea. 
This  fora  too  has  shown  comparatively  few  human  errors  or  huaan 
problems  reported,  even  though  we  are  reasonably  certain  that 
nany  of  then  are  being  overlooked. 

The  reason  for  such  inadequate  huaan  error  reporting  is  that 
almost  all  reporting  systems  are  focused  on  equipaent  failure; 
and  the  reporter  thinks  of  the  fora  in  terms  of  equipaent  fail¬ 
ure  only.  It  is  obvious  that  we  need  sose  system  of  gathering 
human  performance  data  which  ia  distinct  from  systems  directed 
primarily  at  reporting  equipment  breakdown  or  failure. 


It  is  suggested  that  human  perforaance  measurement  be  centered 
around  the  concept  of  s>stea  reaction  tine*  This  reaction  tiae 
ia  defined  in  terms  of  the  time  required  to  perform  each  of 
the  jobs  required  by  the  system.  Our  definition  of  huaan  re¬ 
liability  is  then  the  extent  to  which  actual  performance  devi¬ 
ates  from  theoretically  optimum  performance.  The  deviation 
can  be  expressed  mathematically,  and  one  can  presumably  deter¬ 
mine  the  probability  that  any  given  task  performanes  will 
deviate  a  given  amount  from  the  optiaua.  This  is  in  contrast 
to  the  definition  of  equipment  reliability  which  ia  a  proba- 
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bility  estimate  of  equipment  performing  without  failura  over  a 

•  pec 1 f i ad  time . 

In  system  reaction  tike  we  do  not  deal  directly  with  human  errora 

There  are  several  reasons  for  this: 

1.  Any  human  error  resulting  in  an  equipment  failure  is  autoaati 
cal ly  included  in  the  reliability  index  based  oo  equipaent 
failure  when  that  failure  ia  reported; 

2.  Any  human  error  that  does  not  result  in  equipaent  failure 
serves  aerely  te  increase  the  systea  reaction  tiae  and 
sccoapliahaent  of  the  systea  aission;  such  hnaan  errors  will 
therefore  be  included  autoaati cal ly  under  this  concept; 

3.  The  nuaber  of  huaan  errora  reported  and  the  difficulty  of 
securing  a  representative  sawple  Bakes  it  difficult  to  assess 
huaan  reliability  in  strictly  huaan  error  terms.  How  would 
this  reaction  tiae  aeasure  be  utilised? 

1*  First,  as  part  of  a  job  analysis  Bade,  to  deteraine  particular 
PL  requi rementa ,  determine  how  long  a  particular  job  should 
take  to  accomplish.  It  would  take  entirely  too  long  in  this 
paper  to  go  into  a  eoaplete  dissertation  on  job  or  task 
analysis,  as  it  is  also  called.  Techniques  for  perforaing 
job  analysis  do  exist  and  are  being  applied  daily  by  huaan 
factors  specialists. 

A  job  analysis  can  be  performed  at  any  time,  even  prior  to 
the  job  being  performed  in  the  field.  A  job  analysis  in  an 
R  &  D  situation  presents  soae  difficulties,  since  the  job 
is  in  process  of  being  developed;  but  as  aore  information 
is  gathered,  the  analysis  can  be  refined.  Certainly  the 
analysis  should  be  available  no  later  than  the  start  of 
field  testing,  if  one  is  atteapting  to  assess  field  huaan 
performance;  and  certainly  by  the  tiae  the  systea  is  turned 
over  to  the  military,  if  one  is  to  assess  operational  human 
rel lability. 

1.  The  estimated  time  for  completion  of  a  job  or  taak  should  be 
the  absolute  minimum  time  required  to  perform  the  job  with~ 
out  error.  This  estimate  should  be  based  on  an  analysis 
of  the  factors  involved  in  the  job  plus  one's  experienos 
with  similar  jobs. 

5.  During  field  operations  dsteraine  the  eotual  time  taken  by 
reasonably  skilled  personnel  to  perform  each  Job.  It  Is 
essential  that  personnel  be  trained  to  perform  the  parti¬ 
cular  job  whose  time  one  is  measuring,  and  that  tbs  pro¬ 
cedure  bring  performed  is  accurate.  Tims  measurements 
should  be  mode  by  direot  observation  and  timing  of  person¬ 
nel  performing  the  job  under  standard  operations]  con¬ 
ditions.  Special  arrangements  should  be  mads  to  insure 
that  1 rrtl evanei pi  do  not  interfere  with  and  distort  tha 
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perf oraanct  of  the  teak.  If  at  all  possible,  perforator 
personnel  should  not  know  they  are  being  observed;  however, 
this  aay  in  actual  operations  be  impossible  to  prevent. 

4.  Obviously,  a  single  tine  soaaurenent  on  a  single  individual 
or  group  has  minimal  statistical  validity.  It  is  suggested 
that,  if  at  all  possible,  two  Measurements  should  be  taken 
on  each  of  Sequally  skilled  individuals  or  groups  normally 
performing  the  job.  Circumstances  may,  however,  force  one 
to  use  a  smaller  sample. 

5.  Compare  the  estimated  minimal  (or  criterion)  time  with  each 
actual  performance  time.  The  latter  should  in  each  case  be 
larger  than  for  former  or  at  best  equal  to  the  former.  If 
actual  time  is  less  than  criterion  time,  the  original  job 
analysis  was  performed  incorrectly  and  must  be  repeated. 

6.  The  relationship  between  the  cri ter| pnt fim^  and  the  actual 
time,  expressed  as  a  decimal  ratio  f X  time) one  * 
tentative  measure  of  human  performance  for  a  single  task. 

It  will  be  necessary  to  develop  C/A  ratios  for  each  indi¬ 
vidual  or  group  in  a  sample  and  then  average  each  ratio 
with  other  to  secure  a  mean  ratio  for  the  particular  task 
being  studied.  The  Standard  Deviation  (SD)  of  these  »tio 
might  also  be  secured. 

7.  The  mean  c/a  time  ratio  and  its  SD  for  a  particular  Job 
serves  as  a  description  of  that  job's  human  performance 
reliability.  Where  A  time  is  identical  with  C  time  this 
ratio  is  optimal  (l.OO).  The  greater  the  discrepancy 
between  C  and  A  times  the  lower  is  this  performance  or  re¬ 
liability  indiex.  Where  the  C/A  index  is  low,  it  indicates 
that  human  performance  is  inadequate.  An  investigation 
can  then  be  made  of  the  situation  to  remedy  the  problem. 

In  some  tasks  a  significant  part  of  the  system  reaction  time 
depends  on  the  fixed  equipment  running  time.  For  example,  the 
Atlas  launch  control  system,  once  activated,  has  a  fixed,  in¬ 
variant  running  tine,  regardles  of  any  accompanying  operator 
activity.  In  tasks  of  this  type  it  should  be  possible  to  sub¬ 
tract  the  fixed  equipment  reaction  time  from  overall  perform¬ 
ance  time  and  examine  only  that  part  of  the  performance  af¬ 
fected  by  the  operator.  (4) 

Since  the  system  is  built  up  of  units  of  different  sise- 
l odi vi dual s  and  groups-  it  is  necessary  to  decide  upon  the 
r.ize  of  the  unit  we  will  deal  with.  Logically,  the  size  of 
ihe  unit  should  depend  on  the  number  of  people  programmed  to 
perform  the  job.  Where  a  task  requires  4  men,  we  take  the 
tine  required  to  perform  the  tank,  rather  than  the  times  for 
the  individual  contributions  of  the  four  men.  Any  system 
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contains  N  number  of  individual  Joba  or  tasks.  Assuming  that  we 
have  determined  a  mean  C/A  index  measurement  for  each  job  in  the 
system,  there  is  still  the  necessity  for  combining  these  measure¬ 
ments  in  order  to  seeure  a  total,  unitary  raise  of  human  perform¬ 
ance  for  the  system.  It  might  of  course  be  possible  to  sijply 
average  C/A  indices,  hut  how  accurately  we  average  these  values 
will  depend  on  our  knowledge  of  the  basic  iaterrel mti onshi pa 
among  the  various  jobs  we  have  timed  and  the  weights  to  be  as¬ 
signed  to  each  job.  It  is  to  be  expected  that  our  measurements 
of  human  reliability,  while  they  will  hear  some  as  yet  unspe¬ 
cified  relationship  to  equipment  reliability  indices,  will  be 
measurably  lower  than  these  latter.  Moreover,  precisely  how  much 
human  performance  measurements  could  be  combined  meaningfully 
with  equipment  measures  is  something  that  only  additional  re¬ 
search  can  determine.  The  methods  suggested  are  only  preliminary 
and  require  investigation  to  determine  if  they  can  be  useful. 

The  primary  reason  for  suggesting  them  is  to  provoke  some  interest 
in  the  general  problem  of  measuring  human  performance  within  the 
reliability  context.  Such  a  measure  of  human  performance  could 
be  of  great  value  to  the  reliability  engineer,  since  it  would 
plug  a  significant  hole  in  hia  eetimatea  ar  measnremsnta  of 
system  reliability. 

Summary 

The  major  polnta  that  have  been  made  are: 

1.  Increasing  attention  should  be  paid  to  various  personnel 
logistics  factors  because  they  significantly  affect  system 
rel 1 abi 1 i ty ; 

2.  They  reduce  system  reliability  by  increasing  the  system's 
reaction  time; 

3.  These  personnel  logistics  factors  can  be  anticipated  and 
optimized  by  analyzing  the  various  jobs  to  be  performed 
and  then  asking  which  of  these  factors  are  involved  in  the 
job. 

4.  A  proposed  measure  of  human  reliability  has  been  suggested. 

The  measure  involves  a  comparison  of  criterion  (minimum) 
time  to  correctly  perform  a  task  ano  the  actual  time  re¬ 
quired.  This  ratio  expressed  in  decimal  form  can  be  con¬ 
sidered  a  measure  of  human  reliability. 
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THE  "HIDDEN  DISSUADERS”  OF  MISSILE  SYSTEM  RELIABILITY 
C.  W.  Dean,  American  Institute  for  Research 


The  total  operational  reliability  of  a  complex  system  can  be  no 
better  than  the  reliability  of  itB  most  unreliable  subsystem.  This  is, 
simply,  another  way  of  Baying  that  a  chain  is  as  strong  as  Its  weakest 
link,  and  is  consequently  a  rather  obvious  consent.  For  some  time  now, 
human  factors  specialists  and  others  have  been  looking  at  complex  sys- 
terns  in  a  fashion  which  includes  the  operating  personnel  of  such 
systems  as  an  intrinsic  system  component.  In  addition,  it  has  become 
apparent  in  many  situations  that  this  personnel  subsystem  typically 
exhibits  a  greater  amount  of  performance  variability  than  does  the 
hardware  itself.  If  ve  can  assume  that  this  is  so,  it  follows  that  the 
human  subsystem  exerts  one  of  the  most  important  influences  on  total 
system  reliability. 

Missile  systems  today  are  truly  complex  systems.  This  total  system 
complexity  will  probably  not  become  more  simple  in  the  future,  even 
though  the  hardware  itself  will  be  more  rugged  and  reliable.  There  are 
many  analogies  to  this  in  the  developmental  histories  of  other  complex 
systems,  such  as  aircraft  and  electronic  equipment.  In  these  situations, 
there  have  been  undeniable  improvements  in  component  hardware,  certainly 
in  terms  of  operating  efficiency  if  not  always  with  respect  to  operating 
reliability.  System  complexity,  however,  hew  increased  to  a  large  extent 
because  of  an  ever  increasing  subtlety  in  patterns  of  component  inter¬ 
action  ar.d  feedback  functions.  This  complexity  has  also  brought  about 
many  crucial  problems  in  the  selecting,  training,  and  assigning  of 
personnel  to  operate  and  particularly  to  maintain  such  systems.  These 
personnel  problems  are  certainly  not  solved  today  and  undoubtedly  will 
not  be  completely  solved  in  the  foreseeable  future.  However,  there  is 
at  least  agreement  that  they  do  constitute  problems,  and  many  aspects 
of  the  problems  are  amenable  to  direct  and  well-established  research 
techniques.  It  vouLd  consequently  be  appropriate  here  to  consider 
directly  such  factors  as  the  selecting,  training,  and  assigning  of 
people  as  these  apply  to  the  functioning  of  the  total  missile  system, 
and  as  they  affect  the  Oj^erational  reliability  of  the  total  system. 
However,  there  are  some  sources  and  causes  of  this  personnel  subsystem 
variability  which  unfortunately  are  not  so  clearly  recognised  as  prob¬ 
lems,  are  not  well  documented,  and  certainly  are  not  easily  investigated 
by  usual  techniques.  We  have  chosen  to  emphasize  these  causes  of 
unreliability,  and  have  labeled  them  "the  hidden  dissuaders",  because 
they  are  in  a  cense  the  opposite  of  the  hidden  persuaders  currently 
popular  in  the  advertising  business. 

The  sources  of  the  hidden  dissuaders  are  the  attitudes,  motivations, 
and  other  emotionally  based  fiux  tlons  which  all  humans  inevitably  possess, 
and  which  are  continually  being  reflected  in  external  behavior.  Because 
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they  are  emotional  in  nature ,  the  hidden  di6suaders  do  not  always  follow 
the  rules  of  logic  and  common  sense.  It  might  also  be  added  that  they 
will  seldom  be  eliminated  or  even  reduced  in  degree  by  a  better  human 
engineering  Job  on  a  control  console.  With  these  considerations  in  mind, 
ve  may  now  define  hidden  dissuaders,  and  proceed  with  a  discussion  of 
*hey  might  affect  system  reliability  through  the  kinds  of  behavior 
they  engender  in  the  human  subsystem. 

The  hidden  dissusderB  are  basically  inclinations  to  not  perform  a 
function  well,  not  learn  a  task  properly,  and  not  achieve  success  in  a 
given  situation.  They  are,  furthermore,  reactions  which  are  not  readily 
available  for  common  Bense  analysis  and  direct  remedial  action  because 
the  persons  revealing  these  behaviors  are  often  not  aware  consciously  of 
such  negative  motivations  in  themselves.  In  addition,  supervisors  and 
administrators  may  be  unaware  of  these  negative  reactions  in  their 
people,  because  at  the  observable  level  of  behavior,  the  people  may 
appear  to  be  trying  their  utmost  to  perform  satisfactorily.  Although 
this  may  seem  to  be  a  little  vague  at  the  moment,  let  us  go  further  in 
assessing  how  these  functions,  assuming  their  presence,  may  have  a 
negative  effect  upon  system  reliability.  These  factors  exist  in  all 
varieties  of  activity  encountered  in  life,  but  we  are  interested  speci¬ 
fically  in  the  operation  of  missile  systems,  and  in  uncovering  the  kinds 
of  hidden  dissuaders  which  might  be  peculiar  to  this  particular 
operation. 

The  hidden  dissuaders  likely  to  occur  in  missile  system  operation 
can  generally  be  categorized  on  the  basis  of  their  reference  to  the 
total  personnel  group  or  to  individuals  only.  They  may  also  be  consi¬ 
dered  in  terns  of  tb^tr  occurrence  in  crew  members  as  opposed  to 
supervisory  or  admir ; strati ve  personnel. 

Group  Hidden  Dissuaders: 

Let  us  first  consider  some  of  the  group  reactions  which  might 
negatively  affect  operating  reliability  in  a  miEGlle  system.  To  begin 
with,  it  may  be  a  reasonable  conjecture  that  there  exists  an  almost 
universal  attitude  toward  missiles  which  has  a  negative  tinge  as  a 
result  of  many  factors.  To  many  Americans,  the  idea  of  missiles  and 
rockets,  and  the  rapid  development  in  the  last  few  years  of  entirely  new 
dimensions  in  the  world,  have  produced  tensions  and  anxieties  vhich  even 
the  arrival  of  the  nuclenr  age  did  not  engender.  In  this  sense,  missiles 
may  be  thought  of  today  in  about  the  some  way  that  airplanes  once  were 
considered,  as  rather  dangerous  toys  which  in  the  long  run  will  not 
succeed,  or  more  precisely  should  not  succeed.  This  latter  qualification 
is  quite  important,  since  there  are  large  numbers  of  people  who  have 
rather  strong  feelings  about  the  "rightness”  of  such  things  as  space 
exploration,  virtually  unstoppable  ballistic  weajxjns,  and  so  forth. 

There  have  been  public  statements  by  various  people  to  the  effect  that 
certain  goals  being  sought  and  to  be  sought  through  the  use  of  rockets 
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and  missiles  are  la  themselves  undesirable  or  at  best  unimportant. 

Attitudes  are  expressed  which  question  the  reasons  for  anyone  wanting  to 
find  out  what  Is  on  the  moon  or  the  other  planets  In  the  solar  system, 
even  at  times  Intimating  that  explorations  such  as  these  are  somehow 
un-Christian  and  Immoral.  We  should  not  discount  the  long-range  effects 
of  such  attitudes,  particularly  if  they  prevail  in  large  numbers  of  our 
population,  and  especially  in  Influential  people.  It  is  conceivable 
that  the  sociological  and  cultural  milieu  of  the  present -day  United 
StateB  has  not  as  yet  entered  the  missile  age  wholeheartedly.  The  fact 
that  another  major  power  has  demonstrated  certain  competence  which 
appears  to  exceed  ours  in  this  area  has  created  additional  anxiety 
which  many  find  somewhat  uncomfortable.  The  personnel  available  for 
training  and  assignment  for  missile  system  duties  are  people  who  have 
been  drawn  from  this  cultural  environment,  and  if  a  situation  occurs  in 
which  a  particular  operating  group  is  composed  largely  of  persons  who 
previously  had  little  interest  in  or  basic  emotional  acceptance  of  the 
world  of  missiles,  negative  effects  on  operating  reliability  might 
unintentionally  develop. 

Another  less  general  Kind  of  group  hidden  dissuaders  may  arise 
from  attitudes  and  thinking  patterns  which  are  characteristic  in  certain 
military  circles.  For  example,  it  is  rather  difficult  to  categorise  a 
missile,  particularly  a  major  missile,  in  standard  military  equipment 
terms.  Depending  upon  branch  of  service,  and  type  of  unit  within  a 
Bervice,  there  are  traditional  ways  of  doing  things  and  long -established 
patterns  of  thinking  about  things  which  in  reality  are  not  appropriate 
for  missile  systems,  but  which  have  nonetheless  determined  much  policy 
concerning  them.  The  very  same  miSBile  system  can  be  looked  at  in  an 
entirely  different  fashion  If  one  is  thinking  of  It  as  an  Army  artillery 
weapon  than  If  one  were  considering  it  an  unmanned  air  and  apace  craft. 
While  thlB  in  Itself  may  have  no  effect  upon  system  operation,  there  may 
be  an  unconscious  tendency  to  apply  methods  of  the  past,  no  matter  bow 
effective  they  once  were,  to  problems  of  the  present.  If  the  methods  do 
not  match  the  problems,  solutions  produced  are  something  less  than  Ideal. 

These  generalised  group  attitudes,  feelings,  and  habits  of  thlnklDg 
are  precisely  the  kinds  of  things  which  always  occur  with  the  introduction 
and  development  of  new  concepts  and  new  inventions .  History  has  taught 
us  over  and  over  that  such  attitudes  frequently  do  constitute  a  very 
effective  dissuasion  factor  in  the  development  and  operation  of  such  new 
endeavors.  It  must  be  emphasised  that  in  most  cases  these  group  dissua¬ 
ders  are  based  on  genuine  convictions,  the  care:“ul  application  of 
so-called  common  Bense,  and  with  a  sincere  sense  of  righteous  respon¬ 
sibility.  Unfortunately,  the  circumstances  dictate  that  good  intentions 
are  insufficient,  unless  they  also  happen  to  be  compatible  with  resdity. 

Individual  Hidden  Dlssuadere: 

All  groups  arc  ultimately  collections  of  Individuals,  of  course, 


and  it  1b  the  performance  of  the  individual  and  his  behavior  relative  to 
a  group  endeavor  which  can  be  most  pertinent  in  a  specific  missile  system 
operation.  It  is  possible  to  produce  an  unreliable  missile  system  simply 
by  selecting  the  wrong  people  to  operate  that  system.  Further,  even  if 
the  right  people  are  selected,  the  same  effect  can  occur  if  they  are 
iwA-.  op  trly  trained  and  improperly  supervised.  An  almost  perfect  system 
from  an  equipment  standpoint  can  be  designed,  constructed,  and  delivered 
to  the  operational  setting,  and  subsequently  prove  to  be  extremely  unre¬ 
liable  and  unable  to  accomplish  its  mission  in  an  adequate  fashion.  If, 
cu  the  other  hand,  we  have  selected  personnel  carefully,  trained  them 
competently,  and  assigned  them  to  operational  missile  systems  Judiciously, 
and  if  we  are  futber  sure  that  the  equipment  iu  1  itself  relleble,  we 
may  still  end  up  with  an  unreliable  system.  It  ia  this  latter  kind  of 
situation  which  is  most  puzzling,  since  it  is  so  difficult  to  analyte  the 
cauces  of  unreliability.  And  it  is  precisely  in  this  setting  that  we 
must  look  for  the  hidden  dissuaders  if  the  situation  is  to  be  rectified. 

For  many  of  the  specific  tasks  which  crew  members  in  a  missile 
system  must  perform,  we  may  find  that,  contrary  to  general  opinion,  the 
difficulties  and  complexities  ore  really  not  beyond  the  learning  capabi¬ 
lity  of  available  personnel.  Although  in  some  cases,  there  is  no  question 
but  that  relatively  high  level  persons  are  required  for  effective  perfor¬ 
mance,  there  are  a  large  number  of  duties  associated  with  any  missile 
system  which  in  actuality  do  not  demand  greater  than  average  ability  in 
most  respects.  In  situations  where  system  reliability  is  low  and 
personnel  performance  is  poor,  we  should  consider  not  only  the  abilities 
and  capacities  of  the  people,  but  also  their  motivation.  It  Is  readily 
apparent  that  a  person  who  does  not  really  want  to  da  a  particular  Job 
will,  in  most  circumstances,  simply  go  through  the  motions  of  doing  it 
because  of  external  pressure.  Lack  of  motivation  represents  one  of  the 
most  common  sources  of  poor  personnel  performance.  However,  the  word 
diBBuade"B  implies  a  more  active  negativism  than  would  be  a  product  of 
ordinary  low  motivational  level.  In  the  latter,  it  would  be  expected 
that  evasion  of  extra  work  and  slowness  of  learning  would  be  apparent. 

The  pattern  of  hidden  dissuasion,  however,  1b  found  more  frequently  in 
individuals  who  have  had  strong  positive  motivation,  but  for  one  or 
another  reason  have  become  demotivated.  In  this  situation,  the  element 
of  frustration  has  been  added  which  in  turn  breeds  chronic  resentment, 
a  feeling  of  futility,  and  generalized  disillusion. 

The  foregoing  discussion  points  out  Rome  of  the  basic  areas  in 
which  reliability-destroying  hidden  dissuaders  find  their  nourishment. 

The  unmotivated  and  the  demotivated  will  between  them  succeed  in  lowering 
the  reliability  of  any  complex  system  of  which  they  happen  to  constitute 
a  subsystem.  There  are  certain  unmotivating  and  demotivating  factors 
which  may  be  particularly  relevant  to  missile  system  operations,  and 
perhapo  it  would  be  desirable  to  discuss  a  few  of  these  at  this  time. 

In  almost  all  missile  system  activities,  a  crew  member  must  follow 
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a  written  set  of  procedures,  and  it  is  really  not  necessary  for  him  to 
understand  the  relationships  implied  in  these  procedures  or  for  that 
matter  even  the  effects  of  performing  them.  While  the  proceduralized 
approach  to  the  operation  of  complex  equipment  is  a  thoroughly  eetab- 
n  fib®d  way  of  doing  things  in  our.  culture  today,  it  may  very  well  tend 
to  keep  some  people  at  an  emotional  distance  from  their  tasks.  During 
World  War  II ,  it  was  a  great  Joke  among  soldiers  that  the  Army  always 
does  things  by  the  numbers.  This  procedural  approach  to  simple  tasks 
is  well  as  complex  ones  was  frequently  a  source  of  humor,  scorn,  and 
even  anger  among  Individuals.  It  is  unimportant  that  such  methods  are 
an  objectively  appropriate  way  to  get  a  given  Job  done  quickly  and 
adequately  with  a  wide  variety  of  personnel.  What  is  important  is  the 
fact  that  such  techniques  may  tend  to  keep  the  Individual  at  an  emotional 
distance  from  the  situation,  ard  thus  not  permit  him  to  form  a  personal 
and  to  him  meaningful  identification  with  what  he  is  doing.  A  further 
effect  of  such  procedures  involves  the  unintentional  implication  that 
persons  for  whom  the  procedures  are  designed  are  so  incompetent  and 
Ignorant  that  they  must  be  led  step  by  step  through  a  given  sequence  of 
activities.  For  those  individuals  who  must  be  handled  in  this  manner, 
because  they  really  do  not  have  the  capacity  for  full  understanding 
of  the  system  and  its  many  complex  interactions,  a  demotivating  emotional 
response  can  be  even  more  extensive  because  they  implicitly  realize  that 
they  have  met  their  match.  Such  a  realization,  one  which  few  of  us  care 
to  face.  Inevitably  breeds  hostility  and  negativism.  This  in  turn 
produces  a  sort  of  "who  cares"  type  of  resistance  as  a  chronic  attitude. 
Unfortunately,  none  of  these  things  occur  on  the  surface  where  they  can 
be  handled  by  logical  and  sensible  thinking.  They  are  emotional  in 
nature,  and  in  many  cases  so  unavailable  for  conscious  analysis  that 
little  is  accomplished  by  usual  personnel  handling  techniques. 

An  additional  and  quite  important  source  of  hidden  dissuaders  in 
individuals  is  the  motivational  pattern  which  they  perceive  in  the  group 
leadership.  Thin,  of  course,  is  an  age-old  and  universal  problem,  and 
certainly  not  one  that  is  peculiar  to  missile  system  operation  alone. 
However,  as  was  discussed  previously,  It  is  possible  that  missile  systems 
today  are  more  likeLy  to  reveal  evidence  of  unmotivated  or  demotivated 
personnel  leadership  than  perhaps  some  other  systems  where  the  afore¬ 
mentioned  group  factors  may  not  be  so  extensive.  In  many  operational 
missile  systems,  administrative  and  sometimes  even  technical  leadership 
is  composed  of  people  who  have  had  minimal  interest,  experience  or 
training  in  areas  related  to  missiles.  Through  no  fault  of  their  own, 
theae  persons  are  forced  to  make  sudden  and  extensive  personal  transi¬ 
tions  from  their  previous  career  activities  to  these  new  duties.  Since 
It  often  happens  t.hnt  their  past  experiences  have  little  direct 
applicability  to  the  missile  Held,  it  is  no  surprise  that  they  may 
develop  feeling:;  of  anxiety  and  lack  of  confidence.  Even  where  such 
feelings  are  well  controlled,  frustrations  and  demotivations  can  occur 
which  are  perceived  by  other  members  of  the  group  and  reacted  to  by  the 
development  of  similar  responses. 
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Another  source  of  individual  demotivation  involves  training  methods 
and  programs  that  axe  inadequate  in  scope  or  ineffective  in  results.  We 
might  define  training  according  to  its  real  functional  purpose.  In  this 
ser.^e,  training  might  be  thought  of  as  those  methods  and  techniques  which 
ou  uc'.d  to  adapt  and  modify  one  subsystem,  personnel,  so  that  it  merges 
effectively  on  a  functional  basis  with  another  subsystem,  equipment. 

This  merger  then  results  in  a  total  functioning  system  which  operates 
smoothly  and  efficiently.  If  this  system  merger  turns  out  to  be  incom- 
pxete  or  deficient  a6  a  result  of  inadequacies  in  training,  there  will  be 
functional  conflicts  between  segments  of  the  personnel  and  equipment 
subsystems.  These  tend  to  reduce  the  operational  quality  of  the  total 
cystem.  From  an  individual's  standpoint,  this  situation  can  breed  lack 
of  confidence,  an  incomplete  knowledge  or  sense  of  the  various  Job 
dimensions,  and  even  Ignorance  of  specific  task  demands.  In  many 
situations  involving  other  types  of  systems,  experience  would  tend  to 
o.crcome  many  of  these  factors.  In  missile  systems,  however,  the 
opportunities  for  extensive  practice  in  all  phases  of  the  operation  are 
somewhat  limited.  It  is  seldom  feasible  from  a  number  of  standpoints 
to  provide  frequent  and  extensive  dry-run  practice  with  major  missile 
systems,  particularly  where  continuous  maintenance  of  high  component 
reliability  is  necessary.  As  a  result,  it  is  difficult  for  operating 
personnel  to  achieve  a  great  deal  of  working  experience  with  many  missile 
systems  until  they  have  been  associated  with  such  systems  for  a  consider¬ 
able  length  of  time. 

Finally,  there  are  some  purely  situational  factors  which  contribute 
to  the  development  of  hidden  dlssuaders.  These  usually  involve  personal 
reactions  to  large  numbers  of  frustrating  circumstances.  There  is  a 
certain  amount  of  chaos  in  present  day  missile  system  activities,  and  a 
rather  frantic  trial  and  error  approach  is  often  the  rule  at  all  levels 
of  the  work.  It  1b  possible  that  many  persons  develop  stronger  and 
stronger  feelings  of  frustration  an  they  gain  experience  simply  on  the 
basis  of  their  continuous  encounters  with  large  numbers  of  mistakes  and 
frequent  operational  failures.  The  latter  may  be  particularly  upsetting 
to  individuals  who  have  little  emotional  tolerance  for  the  "building  up 
to  a  big  letdown"  phenomenon,  and  to  persons  who  need  frequent  concrete 
rewards  for  their  hard  work.  As  with  inadequate  training,  these  frus¬ 
trating  experiences  car.  produce  a  chronic  lack  of  confidence,  which  in 
turn  leads  to  the  type  or  negativism  which  characterizes  hidden 
dissuadero. 

So  far,  ve  have  been  discussing  hidden  dissuadero  in  terms  of 
missile  system  administrative  and  operating  personnel.  It  is  conceivable 
that  this  situation  also  exists  in  other  types  of  people  associated  with 
missile  systems.  Designers,  engineers,  bueiress  persons,  and  many  others 
are  no  less  susceptible  to  these  same  emotional  reactions.  Consequently, 
come  of  the  errors  ceen  in  design  and  manufacture  may  arise  from  this 
Bounce.  Here  also,  cuch  factors  or;  lack  of  confidence,  distrust  of  one's 
own  abilitier.,  and  insecurity  resulting  from  frustrations  may  breed 
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demotivation.  In  situations  where  problems  are  extremely  difficult  and 
success  rewards  infrequent ,  thi3  sort  of  development  may  be  expected. 

The  Diagnosis  of  Hidden  Dlssuaders: 

A  thorough  analysis  of  any  given  missile  system  operation  will 
reveal  certain  symptoms  which  suggest  the  presence  of  hidden  dissuaders 
in  the  personnel  subsystem  as  a  reliability-reducing  element.  If  a 
given  missile  system  has  proven  reliable  in  some  situations  and  with 
some  operating  groups,  and  unreliable  in  others,  we  have  a  potential 
symptom.  If,  over  a  period  of  time,  this  pattern  shows  itself  to  be 
consistent,  we  can  be  almost  sure  that  hidden  dissuaders  are  at  work 
in  the  less  successful  group. 

In  situations  where  reliability  problems  in  a  missile  system  appear 
to  defy  all  objective  and  sensible  analysis,  no  matter  how  thoroughly 
conducted,  some  thought  should  be  given  to  the  possibility  that  these 
more  subtle  influences  are  involved.  This  particular  symptom  will  often 
be  indicative  of  the  system  development  stage  at  which  hidden  dissuaderB 
are  or  have  been  operating.  A  missile  system  which  has  never  worked 
properly,  and  which  has  defied  all  efforts  to  improve  its  reliability, 
may  be  the  victim  of  hidden  dissuaders  in  research,  design,  or 
construction  personnel. 

The  Elimination  of  Hidden  Dissuaders: 


If  it  is  found  in  a  given  missile  system  that  overall  operating 
reliability  is  being  adversely  affected  by  the  kinds  of  human  subsystem 
factors  we  have  been  discussing,  there  are  some  methods  which  can  be  used 
to  reduce  or  eliminate  such  problems.  Furthermore,  the  taking  of  certain 
measures  may  help  to  prevent  the  development  of  hidden  dissuaders  in 
future  missile  systems. 

One  of  the  first  steps  to  be  taken  in  achieving  these  ends  is  to 
provide  some  sound  structure  in  the  personnel  subsystem.  Within  the 
limits  of  available  personnel,  selection  procedures  should  be  aimed  at 
acquiring  a  reasonably  well  balanced  operating  group.  This  means  that 
various  ability  levels,  appropriate  to  the  different  task6  and  duties, 
should  be  cought.  A  system  which  is  overloaded  with  highly  abl”  people 
may  oe  Just  as  prone  to  personnel  difficulties  as  one  in  which  general 
ability  is  too  low.  A  thorough  analysis  of  tasks  and  duties  involved 
in  the  specific  missile  system  should  precede  personnel  selection  and 
assignment . 

A  second  step  in  attempting  to  eliminate  or  prevent  hidden  dlsoua- 
ders  involves  the  training  ran thoJs  to  be  used  with  personnel  selected. 
When  Dpecial  training  equipment  is  utilised  in  thi6  endeavor,  it  is 
neceoooxy  to  determine  whether  the  equipment  itself  and  its  manner  of 
utilization  axe  producing  the  desired  training  results.  Keeping  in  mind 


the  system  merger  goal  of  training,  there  should  be  much  attention  given 
to  ways  in  which  the  goal  can  be  achieved  with 'a  minimum  waste  of  time 
and  effort. 

A  very  large  portion  of  potential  or  already  existing  hidden 
dissuaders  can  be  eliminated  by  a  basic  change- in  leader  attitudes. 

Careful  selection  of  leaders  can  take  care  of  some  of  these  problems, 
but  in  other  cases  considerable  reeducation  may  be  necessary.  The 
responsibility  for  this  reeducation  lies  to  a  large  extent  with  the 
leader  himself.  He  should  make  a  serious  effort  to  determine  whether 
his  own  feelings  and  attitudes  are  the  kinds  which  are  likely  to  produce 
hidden  dissuaders  in  himself  and  others.  If  they  are,  he  should  try  to 
find  practical  ways  of  altering  them.  Such  self-analysis  unfortunately 
has  limited  usefulness,  since  all  persons  rationalise  their  own  attitudes 
and  behavior,  not  dishonestly,  but  as  an  unconscious ly  motivated 
self -protective  device.  Whether  we  like  it  or  not,  on  objective  apprai¬ 
sal  from  a  disinterested  source  is  necessary  occasionally,  if  constructive 
solutions  to  these  particular  problems  are  to  be  found. 

Finally,  greater  awareness  of  the  sources  of  variation  in  human 
behavior,  other  than  usual  and  obvious  ones,  should  become  part  of  the 
knowledge  pool  of  administrators.  In  addition,  further  research  is 
needed  in  methods  of  compensating  for  human  variability,  which  is  after 
all  inevitable,  rather  than  attempting  to  completely  eliminate  it.  Some 
refined  concepts  in  this  area  are  presently  under  development.  It  Is 
intended  that  improvements  considerably  beyond  those  accomplished  to 
date  by  human  engineering  and  human  factors  work  will  be  made  in  the 
personnel  -  equipment  system  interaction.  It  is  not  appropriate  here  to 
go  into  details  of  this  development,  but  an  example  might  suggest  its 
direction.  An  airline  pilot,  when  asked  which  is  his  favorite  airplane, 
replied  "Why  the  old  DC -3  of  course.  The  reason  I  like  the  ship  is 
because  I  can  make  a  lot  of  mistakes  when  flying  it,  and  it  helps  compen¬ 
sate  for  my  errors.  In  a  way,  you  might  say  that  it  can  actually  think 
a  little  bit  and  take  over  for  me  when  I  am  wrong.  Most  of  our  newer 
high  performance  aircraft  are  much  too  'stupid'  in  this  regard,  and 
everything  depends  upon  my  performance  in  a  tricky  situation."  In  the 
near  future,  we  hope  to  provide  methods  by  which  complex  systems,  such 
as  missile  systems,  con  be  greatly  increased  in  efficiency,  performance, 
and  reliability  by  means  of  a  new  approach  to  the  personnel  -  equipment 
interrelationship. 

Summary ; 

We  have  been  discussing  t.ie  negative  influence  of  subtle  human 
emotional  reactions  on  missile  system  reliability.  These  reactions  have 
been  called  "hidden  di r-suaders" ,  since  they  Eire  seldom  clearly  revealed 
by  usual  methods  of  personnel  handling  find  Investigation.  Hidden 
dissuaders  may  arise  from  several  sources: 
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1. 

Mixed  feelings  which  nay  exist  rather  generally  in  the  United 
States  today  about  missiles  and  rockets,  particularly  with 
regard  to  their  eventual  purposes  and  uses. 

2. 

differences  in  viewpoint  as  to  the  proper  categorisation  of  a 
missile  as  a  weapon. 

1 

3. 

Non-motivating  and  demotivating  influences,  such  as  frustration, 
disillusion,  frequent  failure,  over-proceduraliiation,  leader 
attitudes,  and  ineffective  training. 

Hidden  dissuadei'6  are  probably  operating  in  missile  systems  which: 

1.  Over  a  period  of  time  show  some  non-equipment -based  pattern  in 
their  reliability  problems. 

2.  Have  never  worked  well,  and  seem  to  defy  all  attempts  to  analyse 
the  sources  of  unreliability. 

Suggested  methods  for  reducing  or  eliminating  the  hidden  diasuaders 
include : 


1.  Establishing  the  system  personnel  structure  carefully  and  on 
the  basis  of  factual  knowledge  of  Job  demands. 

2.  Providing  effective  training  so  that  personnel  confidence  will 
be  improved. 

3.  Selecting  or  producing  leaders  who  do  not  themselves  instill 
demotivation  in  subordinates. 

4.  Looking  for  new  approaches  to  personnel  -  equipment  system 
interactions.  A  new  development  in  this  area  is  currently 
being  worked  out  which  may  help  to  solve  many  specific 
system  problems  in  the  future. 
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THE  DEMONSTRATION  OF  EQUIPMENT  RELIABILITY 
THROUGH  ACCELERATED  LIFE  TESTING  TECHNIQUES 
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Since  data  associated  with  variable  type  sampling  is  readily 
analysed  by  use  of  known  and  tried  techniques  only  data  con¬ 
cerned  with  attribute  type  sampling  will  be  discussed. 

Although  it  is  of  primary  interest  to  know  the  ’’critical  stim¬ 
ulus"  it  is  often  quite  difficult  to  obtain  this  in  practice; 
especially  if  we  are  dealing  with  animate  samples.  There¬ 
fore,  it  behooves  us  to  run  parallel  experiments,  on  such 
items  as  munitions  or  strength  of  materials. 

The  decision  as  to  whether  a  lot  meets  the  desired  require¬ 
ment  (e.g.  small  fraction  failing)  under  specified  operating 
conditions  will  be  based  upon  inferences  from  a  sample.  Which 
method  is  chosen  will  depend  on  whether  it  gives  the  maximum 
reliable  information  from  a  minimum  of  data  Let  us  consider 
the  following  types : 

A.  Tests  conducted  only  at  the  stimulus  called  out 
in  the  specifications. 

B.  Tests  conducted  at  some  stimulus  more  stringent 
than  that  required  in  the  specifications. 

C.  Tests  conducted  at  two  or  three  stimuli,  all  of 
which  may  be  different  from  the  one  required  in 
the  specifications. 

D.  Complete  run-down  tests  -  All  stimuli,  equally 
spaced>  from  1007.  failing  to  none  failing. 

In  actual  industrial  or  Government  practice,  many  tests  fall 
into  category  A.  This  is  due  in  a  large  part  to  the  manner 
in  which  the  specifications  are  written;  such  as  105A.  How¬ 
ever,  tests  of  this  kind  are  generally  inefficient,  since  if 
small  samples  are  used,  a  high  degree  of  confidence  that  the 
lot  has  the  high  quality  called  for  by  the  specification  can¬ 
not  be  attained  even  if  all  the  items  in  the  small  sample  pass 
the  test  satisfactorily.  To  attain  a  high  confidence  level  that 
the  submitted  lot  actually  has  the  specified  quality  very  large 
samples  are  necessary. 
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The  following  Cable  will  serve  to  illustrate  the  probability 
of  accepting  material  of  a  given  quality  based  on  samples  of 
20  and  100  where  the  basis  of  acceptance  is  finding  no  defects 
in  the  submitted  sample. 


TABLE  1 


b ample Probability  (of  accepting  lots  whose  percentage 
Size _ defective  iB  as  shown  in  the  table) _ 


.01 

.05 

.10 

.50 

.90 

.95 

.99 

20 

217. 

147. 

117. 

3.57. 

.57. 

.257. 

.057. 

100 

4.57. 

2.97. 

2 . 37. 

.777. 

.17. 

.057. 

.017. 

This  table  shows  that,  based  on  samples  of  20  or  even  100,  it 
is  almost  impossible  to  guarantee  the  acceptance  of  lots  hav¬ 
ing  a  quality  level  of  less  than  .17.  defective.  As  shown 
above  907.  of  lots  which  have  .17.  defectives  would  be  accepted 
on  the  basis  of  no  failures  in  a  sample  of  100,  Actually,  to 
ensure  the  rejection  of  lots  containing  more  than  .17.  defective 
957.  of  the  time,  would  require  a  sample  of  3250.  To  ensure  the 
rejection  of  such  lots  997.  of  the  time  would  require  testing  a 
sample  of  5000  and  finding  no  defects.  This,  of  course,  leads 
directly  to  the  question:  How  can  we  ensure  the  rejection  of 
inferior  lots  and  still  use  a  reasonable  small  sample? 

The  answer  to  this  question  is  contained  in  tests  falling  into 
Category  B,  that  is,  conduct  the  test  at  a  stimulus  level  which 
will  produce  a  large  percentage  of  failures,  say  207.  to  507.. 

This  method,  however,  can  be  used  only  if  an  actual  distribution 
function  exists  connecting  the  percentage  failing  with  the 
applied  stimulus.  Without  the  existence  of  this  relationship, 
data  collected  at  the  507.  failure  point  would  be  of  absolutely 
no  value  in  attempting  to  predict  any  results  at  a  stimulus 
giving  a  small  percentage  failure  rate. 

In  order  to  determine  that  stimulus  which  should  give  507. 
failure  it  i6  necessary  to  perform  an  initial  rundown  test  on 
samples  drawn  from  a  standard  lot.  A  standard  lot  may  be  one 
that  has  given  satisfactory  performance  and  meets  the  speci¬ 
fication  requirements . 
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Making  one  assumption,  namely;  'critical'  stimuli  are  normally 
distributed  with  the  standard  deviation  for  all  lots  being 
equal;  then  increased  severity  tests  conducted  at  a  single 
stimulus,  giving  507,  failures,  (in  reference  lot)  are  sensi- 
ti\  -  i.-.dieator  of  significant  differences  among  lots  at  the 
specification  stimulus  even  with  relatively  small  samples. 

In  order  to  get  some  idea  of  the  efficiency  of  an  increased 
severity  test,  let  us  consider  the  following  example: 

1.  Let's  call  acceptable  any  lot  having  .17.  failures 
in  a  standard  lot. 

2.  Let's  call  unacceptable  any  lot  having  17,  (or  more) 
failures  at  same  stimulus  as  above. 

3.  Let's  call  marginal  those  lots  which  fall  in  be¬ 
tween  .17.  and  17.  failures. 

4.  The  producers  risk,  or  the  risk  of  rejecting  a 
lot  having  .17.  failures,  shall  be  .05. 

5.  The  consumers  risk,  or  the  risk  of  accepting  a 
lot  having  17.  failures,  shall  be  .01. 

Based  on  the  above,  it  can  be  shown  that  even  under  the  best 
possible  sampling  methods  an  average  sample  size  of  600  will 
be  needed  in  order  to  reach  ?  decision  concerning  the  accepta¬ 
bility  of  the  lot. 

However,  suppose  we  translate  the  problem  involving  the  dis¬ 
crimination  between  lots  which  give  .17.  and  17,  failures,  re¬ 
spectively  at  che  stimulus  giving  .17.  failure  in  the  standard 
lot,  into  one  involving  a  comparison  carried  out  at  a  stimulus 
giving  507.  failures  in  the  standard  lot,  then  what  sample  sizes 
are  required  in  order  to  arrive  at  a  decision  with  the  risks 
stated  above’ 

Making  the  assumption  stated  previously,  namely;  "critical" 
stimuli  are  normally  distributed  with  the  same  standard 
deviation  Or),  then  the  critical  stimulus  giving  507.  failures 
in  a  lot  having  .17.  failures  at  the  specification  stimulus  is 
3.090"' away  from  that  stimulus.  However,  application  of  the 
stimulus  giving  507,  failure  in  a  lot  having  .17.  failures  in 
the  standard  lot  results  in  78%  failures  in  the  second  lot. 
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Figure  1  shows  this  graphically,  where  Curve  I  is  Distribution 
of  critical  stimuli  in  the  standard  lot  having  .17.  failures 
at  specification  stimulus  and  Curve  II  is  the  Distribution  of 
critical  stimuli  in  lot  having  1%  failures  at  the  specification 
3ti'n,»i.u«« .  What  has  been  done  then  is  to  convert  a  comparison 
between  lots  having  17.  and  .17.  failures  at  the  specification 
stimulus  into  one  between  lots  having  5071  and  787.  failures 
respectively  at  the  "increased  severity"  level.  As  a  result 
nf  this  transformation  we  shall  now  require  to  take  on  the 
average  a  sample  of  only  32  (or  less)  in  order  to  reach  a 
decision  of  acceptability.  Thus,  the  required  sample  size 
has  been  reduced  from  600  to  32  by  the  method  of  Increased 
severity.  We  wish  to  point  out,  however,  that  the  large  saving 
in  sample  size  results  from  assuming  a  constant  and  known 
standard  deviation  for  the  critical  stimuli  and  that  these 
stimuli  are  normally  distributed. 

In  actual  practice,  the  value  of  the  stimulus  giving  507.  fail¬ 
ures  is  not  constant  and  neither  is  their  standard  deviation. 
Because  of  this,  tests  of  increased  severity  conducted  at  only 
one  stimulus  can  give  misleading  and  erroneous  results.  For 
example,  if  the  standard  deviation  of  critical  stimuli  associ¬ 
ated  with  an  explosive  mixture  is  large,  than  the  explosive 
mixture  is  undesirable;  since  there  is  a  chance  that  it  may 
either  fail  to  explode  at  the  stimulus  designated  in  the  speci¬ 
fication  or  that  it  may  explode  prematurely. 

Because  of  this  fact,  tests  of  type  C  were  evolved;  that  is 
increased  severity  tests  at  two  or  more  stimuli.  When  two 
stimuli  are  used  it  is  customary  to  choose  them  at  about  the 
207.  and  807.  failure  points.  Actually,  the  best  estimates  are 
achieved  with  the  67.  and  947.  failure  points.  However,  trying 
to  achieve  these  percentages  covild  result  in  07.  and  1007.  values 
being  obtained  thereby  nullifying  the  method.  Let  us  desig¬ 
nate  as  the  true  fraction  failing  at  stimulus  ''Y^,  then  the 
can  be  represented  as  integrals  of  a  normal  distribution 
from^  ,  to  co  • 
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Therefore,  what  has  to  be  determined  are  the  estimates  of  the 
true  mean, yA  and  true  standard  deviation,  <7~ .  These  estimates 
are  determined  experimentally  and  labeled  ?  and  s  respectively. 
The  following  are  the  parameters  to  be  evaluated: 

^  :  observed  fraction  failing  at  yLL 

*1/1  z  true  fraction  failing  at  *y<! 

z  sample  estimate  of  true  mean  critical  stimulus 

«  true  mean  critical  stimulus 

sample  estimate  of  true  standard  deviation 

(T~  z  true  standard  deviation 


On  the  assumption  of  a  normal  distribution,  and  by  creating  the 
experimentally  determined  and  P-  as  the  true'JT’  and  J£and 
associating  with  each^Ta  where i  and  where  values 
of  ti  V  are  to  be  found  in  any  standard  tables  of  normal  areas, 
the  following  simultaneous  linear  equations  are  obtained: 


(2)  ~ 

r2  =  -**■-*- 


for  x  and  s  as  follows 


-y  '  •—  • 

(3)A  *  /  / 


which  may  be  solved 


where  d  s  ^ 


A 


1 


This  is  known  as  the  probit'  method  of  estimating  the  para¬ 
meters  of  the  normal  distribution.  By  carrying  out  the  test 
at  two  stimuli,  the  two  lots  are  compared  as  to  their  means 
and  their  standard  deviation  or  given  symbolically. 


(5) 


+  H 


It  we  use  three  stimuli,  the  third  stimulus  P.  will  serve  as 
a  check  on  the  validity  of  the  assumption  of  normality.  How¬ 
ever,  if  we  use  more_  than  two  stimuli  for  sensitivity  tests, 
then  the  equations  X  and  s  shown  above  will  no  longer  hold, 
instead,  x  and  s  will  have  to  be  determined  by  means  of  least 
squares . 

It  is  therefore,  a  logical  extension  of  the  two  and  three 
stimuli  test  to  the  complete  run-down  test,  for  this  test 
the  stimuli  used  range  from  the  stimulus  for  which  none  of 
the  objects  are  affected  to  tie  stimulus  for  which  all  of  the 
objects  tested  as  affected. 

A  simple  visual  manner  of  summarizing  the  various  methods  dis¬ 
cussed  above  is  by  means  of  operation  characteristic  curves, 
abbreviated  OCC.  By  means  of  these  curves  a  comolete  picture 
of  an  inspection  plan  or  an  experimental  design  is  obtained. 

Let  us  consider  the  example  discussed  previously,  that  is  if 
in  a  sample  of  20,  if  no  defects  occur,  we  accept,  otherwise 
reject.  The- same  plan  can  be  used  with  a  sample  size  of  100. 

The  0C  curves  for  these  are  shown  in  Figure  2.  These  are  tests 
run  at  the  specification  stimulus.  Let  us,  in  addition  plot 
the  OC  curve  for  an  increased  severity  test  on  the  same  graph. 
Inspection  of  the  curves  shows  that  the  -only  difference  be¬ 
tween  curves  I  and  II  are  that  curve  B  is  steeper  and  shifted 
to  the  left  of  curve  I.  The  meanings  attached  to  these  dif¬ 
ferences  are  as  follows: 

1.  The  shift  to  the  left  shows  that  the  chance  of 

accepting  material  is  less  which  is  to  be  expected,  since  the 
probability  of  finding  no  defects  in  100  samples  is  less 
than  in  20  samples .  .  - 

2.  The  steepness  indicates  that  the  risk  of  accepting 
bad  lots  or  rejecting  good  ones  are  decreased. 

Curve  III,  tested  at  the  stimulus  where  507,  fail  shows 
a  marked  improvement  without  sample  size. 

There  are  numerous  cases  in  manufacturing  processes  where  it 
is  extremely  Important  to  have  no  defectives.  However,  since 
this  is  impossible  to  achieve  with  any  sampling  plan  it  resolves 
itself  into  a  problem  of  assuming  a  very  small  risk  of  accepting 


material  with  .0017.  defective  or  more.  This  type  of  condition 
occurs  in  the  manufacture  of  ballistic  weapons  where  costly 
items  may  fail  completely  because  the  primer  or  detonator  did 
not  function  properly. 

Let  us  assume  for  illustrative  purposes  that  we  wish,  say  507. 
assurance  that  a  lot  contains  nb  more  than  3  defective  in  10 
million.  With  an  increased  severity  test,  either  a  2  stimuli 
or  complete  rundown  type,  we  can  test  50  samples  at  each  stimu¬ 
lus  in  the  complete  rundown  test  or  about  200  at  each  stimulus 
in  the  case  of  the  two  stimuli  test.  Figure  3  shows  a  plot  of 
the  percentage  failing  against  the  probability  of  acceptance. 

On  the  same  graph  is  plotted  the  results  of  the  best  single 
stimulus  test,  that  at  the  specifcation  stimulus.  Figure  4 
shows  the  amount  of  testing  required  using  the  specification 
stimulus  which  in  this  case  would  run  in  the  neighborhood  of 
100,000  items  as  compared  to  about  400  with  an  increased 
severity  test. 

Another  advantage  obtained  through  increased  severity  testing 
is  the  ability  to  set  up  '‘safety-factors"  for  inspection  tests. 
This  generally  cannot  be  done  if  a  single  stimulus  type  test 
is  used. 

Figure  5  is  based  on  the  sampling  plan  shown  in  Figure  4A  and 
plots  the  probability  of  acceptance  against  the  maximum  stimu¬ 
lus  where  only  3  defectives  in  10. million  occur.  From  the 
graph,  it  is  seen  that  practically  no  lot  will  be  accepted 
whose  maximum  stimulus  exceeds  the  specification  stimulus  by 
more  than  1.25  0-. 

Let  us  consider  the  following  problem: 

A  tensile  specimen  is  being  tested  and  it  is  desired  that  no  more 
than  1  in  1000  will  withstand  a  load  of  4000  lbs.  Twenty  sam¬ 
ples  where  tested  at  each  of  seven  loads  ranging  from  500  lbs 
to  3500  lbs  at  500  lb.  increments.  The  results  are  tabulated: 


Load 

- 

X  (lbs) 

No.  Tested 

No.  Passing 

Passing  -  P 

X1 

a 

500 

20 

20 

pi 

1.00 

X2 

S 

1000 

20 

19 

P2 

.95 

X3 

a 

1500 

20 

18 

P3 

.90 

X4 

m 

2000 

20 

14 

P4 

.70 

X5 

= 

2500 

20 

4 

1 

P5 

o 

CM 

• 

- 

3000 

20 

1 

p6 

.05 

*7 

• 

3500 

20 

0 

p7 

0 

Using  the  following  form  to  simplify  calculations,  where 


S'-  _ 

*7  L  s 

fraction  passing 

1 1 

1  -  P2 

m 

Std.  Dev.  factor 

f  = 

interval 

of  load 

Xi  * 

loads 

3** 

interval 

all  passing 
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a 

m 

m 

HU 

BHP 

lv.00 

.95 

1 

.048 

.95 

21.16 

1.02 

.90 

3 

.09 

2.70 

6.76 

.61 

.70 

5 

.21 

3.50 

.36 

.08 

* 

.20 

.16 

1.40 

mam 

1.96 

.31 

3000 

.05 

.048 

.45 

mm 

11.56 

.55 

2.3 

’ 

.56 

9.0 

m 

2.57 

• 

r 

zr 

-  sz 

£4.1^ 

-  7o 

ft*. 

- /to* 

2, 

=  TSi \ 

M 

V*  *7 

'i/ajoi 

'Sf  * 

73*° 

n 

VS" 

cr~ 

=  /•/£ 

% 

■^1/5  &\ 
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*  6~40 

5F- 
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i 
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/cr 

The  parameters  multiplied  by  -f ,  the  interval  s ize,  connects  the 
computational  value  to  the  actual  value. 

When  the  intervals  arc  a  Trace ion,  change  the  fraction  to  its 
decimal  equivalent  and  proceed  as  above.  As  can  be  seen,  it 
is  simpler  to  find  all  our  parameters  without  using  the  inter¬ 
val  size  in  computation  but  rather  to  adjust  for  it  at  the  end  • 

of  the  computation. 

At  the  95%  confidence  level,  the  true  mean  critical  stimulus,/^ 
lies  between  1933  and  231>'  lbs  and  at  the  99%  confidence 
level^/V  would  lie  between  1931  and  2369  lbs. 

However,  our  interest  lies  in  having  at  most  1  in  1000  speci¬ 
mens  withstanding  4000  lus  load. 


90 


AC  the  99.97.  level 


X  +  3.09s  .  2150  +-  1670  .  3820 

and  s2  9g  9  *  sf  *  (3.09)2/.  13225 

giving  899  9  =  US 

The  probability  of  getting  a  sample  value  3820  from  a  popu¬ 
lation  having  a  mean  of  4000  in  aporoximately  1.74  Otway 
from  it  and  we  can  state  that  the  test  is  significant  at  the 
57.  level;  that  is  there  is  less  than  1  chance  in  20  of  get¬ 
ting  this  value. 

Since  the  aim  of  a  test  program  is  not  solely  to  reduce  sample 
size  but  also  to  avoid  excessively  restrictive  assumptions,  it 
is  felt  that  tests  of  increased  severity  strike  the  balance. 

This  was  borne  out  earlier  in  the  discussion  where  it  was  shown 
that  600  samples  were  necessary  to  insure  rejection  57.  of  the 
time  of  lots  having  .17.  defectives  when  testing  at  the  speci¬ 
fication  stimulus.  We  saw  that  32  samples  were  required  test¬ 
ing  at  the  507.  critical  stimulus.  However,  certain  assumptions 
had  to  be  made  namely,  the  critical  stimuli  are  normally  dis¬ 
tributed  for  all  lots  and  have  the  same  standard  deviation. 

In  actual  practice  using  the  complete  rundown  method,  the  num¬ 
ber  of  samples  required  will  lie  somewhere  in  between  32  and  600. 

However,  the  gain  in  information  about  the  distribution  of  the 
critical  stimuli  obtained  will  more  than  compensate  for  the 
increased  sample  size  compared  to  the  single  stimulus  increased 
severity  test.  Sometimes,  due  to  factors  beyond  our  control, 
it  is  not  feasible  to  test  at  equally  spaced  intervals. 

If  the  intervals  are  not  equally  spaced,  a  first  difference 
method  can  be  used  to  estimate  the  parameters.  For  purposes 
of  illustration  a  primer  testing  problem  will  be  used. 
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where  q^  s  fraction  misfiring  at  height 


For  ease  of  computation  we  tabulate  the  data  as  follows: 


Ji  *•  v ' 

ZV 

■zl  -  ■? 

13 

.10 

-2 

-1.28 

-1.69 

14 

.60 

-1 

.25 

-  .16 

15 

.70 

0 

.52 

.11 

15.5 

.80 

.  5 

.84 

.43 

16 

.80 

1 

.84 

.43 

16.5 

.90 

1.5 

1.28 

.87 

1. 


K 


z  assumed  mean ;  X,  z  computed  mean 


(/ ~ *r  ^ 


TTT-  i )(ui  -{)-*/■  3  f 


X  ~~  ?  t 


t*¥f 


2 .  r; 


obtained  from  "t"  table 


PROBIT  METHOD 


The  "Probit  Method"  of  analysis  is  actually  a  means  of  trans¬ 
forming  normal  data  where  measurement  is  based  upon  only 
wnether  the  measured  characteristic  is  above  or  below  a  cer¬ 
tain  stimulus.  The  most  common  application  of  this  method  of 
analysis  is  in  the  biological  sciences  where  lethality  of 
varying  dosages  of  drugs  might  be  studied.  In  the  field  of 
ballistics  it  can  readily  be  applied  as  well,  even  though 
it  might  be  felt  that  the  stimulus  could  be  gradually  in¬ 
creased  until  detonation  occurred.  However,  any  impulse 
applied  to  the  primer  that  doesn't  fire  it,  undoubtedly 
changes  its  sensitivity  characteristics.  Thi9  method  depends 
on  fitting  a  straight  line  to  the  observed  data.  However, 
since  the  variance  of  the  independent  variable  is  not  con¬ 
stant  but  depends  on  "t"  the  usual  means  of  fitting  a  straight 
line  can  not  be  used.  Instead  a  method  of  successive  approxi¬ 
mations  is  employed. 


A  method  of  obtaining  a  first  approximation  for  equally  spaced 
intervals  can  be  accomplished  from  the  data  accumulated  from 
a  "run  down"  experiment.  Tabulating  results  of  primer  drop 


test  data  gives: 


f  ’w. 

;  V, 


From  this  table  we  get  £  s  18.599 

=  1.989 


Using  t  ■  h  -  x 


t  =  h  -  18599 

059 


h  s  x  +  18 


a  -  -  .599  =  -  .301 
1.989 

•  x  +  18  -  18.599 

- nws — 


t  ■  x  -  .590 
17989 


. 301 *#■  . 503x 

b  «  1  =  .503 

1.989 


a  useful  means  for  obtaining  a  first  approximation  is  the 
graphic  method  since  it  can  be  used  for  equally  or  unequally 
spaced  intervals. 


Given : 


Charges 

(grains) 

Number 

fired 

Number 

successes 

7*Successes 

P 

"t"  from 
Table 

60 

3 

0 

.00 

•1.28 

12G 

3 

2 

.67 

.44 

180 

3 

2 

.67 

.44 

240 

3 

3 

1.00 

1.28 

300 

3 

. 

l.Go 

1.28 

When  the  interval  is  not  equal  (5,5,10,5,3)  etc....  ake  the 
W  value  from  the  tabic  and  multiply  by  the  n  or  a  reduction 
of  it . 

From  first  approximation  t  .  .44  +  .84^'  (See  Graph  Fig.  7) 


1  -  m 


t 

Q 

Z+ 

7 

-2 

-1.24 

1.00 

.8925 

-.1075 

.1849 

.3565 

-.5314 

-.7130 

mm 

-1 

-  .40 

.33 

.6554. 

.3254 

.3633 

.6005 

.8833 

-.6005 

.5303' 

0 

.44 

.33 

.3300 

.0000 

.3621 

.5932 

.0000 

.0000 

.0000 

1 

1.20 

.00 

.1003 

.1003 

.1758 

.3428 

.5715 

.3428 

.1956 

2 

2.12 

.00 

.0170 

.0170 

.0422 

.1064 

.4028 

.2128 

.0428 

u _ : 

MU 

awfpi 

"75616 

q  :  1-P 

"Q",  "Z",  ,-W"  are  found  in  "Tables  for  Analysis  of  Experi¬ 
mental  Bata",  when  "t"  is  negative,  subtract  Q  value  found 
in  table  from  1.0000 


"Z"  and  "VJ"  is  taken  from  the 


X 

xcl 


7~ 

X  '  2^ 
glv/xy 
z^yx 


=  -.379 

b  .1653 

=  .2129 
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The  final  approximation  is  determine'  if^and</dis  less  than  .01 
of  first  approximation.  Therefore,  the  final  approximation  is 
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To  estimate  the  error  associated  with  these  parameters  compute: 


where  k  is  the  factor  for  reducing  the  sample  size, ^  and  1  is 
t’e  scale  reducing  factor.  Here  k  mj£  >  .1 

*5 * -a,  - j-  i y  -± 

Another  procedure  for  statistical  analysis  of  sensitivity  data 
is  one  that  was  used  extensively  by  the  Explosives  Research 
Laboratory  at  Bruceton,  Pa.  This  method  is  known  as  the  "stair¬ 
case"  or  "up  and  down"  method,  and  quite  often,  as  the  Bruceton 
method.  The  difference  between  this  method  and  the  ones  previ¬ 
ously  described  lies  in  the  fact  that  this  method  does  not  use 
a  fixed  sample  size  at  each  height.  Instead,  the  height  of 
drop,  is  changed  by  a  fixed  increment  after  each  single  speci¬ 
men.  If  the  explosive  fired  at  the  height  tested,  then  for 
the  next  test  the  height  is  reduced  to  (on  the  other 

hand,  if  it  fails  to  fire,  the  height  is  increased  by  this 
fixed  increment  to  h^)  This  method  insures  a  great  number 
of  tests  being  conducted  at  or  near  the  mean  height  of  fire, 
that  is,  at  the  height  at  which  fifty  percent  fire.  If  we 
let  +  denote  firing  and  8  misfiring  a  plot  of  a  test  could  look 
as  follows 


where  n  is  the  height  and  d  is  a  iixed  difference.  The  initial 
testing  height  is  usually  chosen  on  the  basis  oi  past  experience. 


1  -  197 


The  important  advantages  of  this  method  are: 

1.  An  accurate  estimate  of  the  mean,  stronger  than 
run-down  test. 

2.  Computation  of  mean  and  variance  is  relatively 
s imp le . 

3.  Reduced  sample  size. 
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The  meaning  of  Che  terms  are  as  follows: 

c  ■  normalized  heights  of  lowest  line  ;  .923 
N  =  number  firing  or  misfiring  (lesser) s  50 
d  •  difference  between  log  h^ 
i  =  reduced  height 

number  firing  at  reduced  height  i 
Suitmarizing  the  information  from  Fig.  6  we  have 
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Since  the  +'s  and  0's  are  equal  in  number,  we  will  use  the 
formula  for  the  mean  based  on  -^'s  in  this  case  --  although 
either  could  be  used. 

This  gives 

ro  =  .928  +  .093  (100/50 

z  1.068 

and  M  =50  (218)  -  (100)2 
«  360 

From  Table  I  the  value  of  s  corresponding  to  a  value  of  M 
.  .360  is  s  .  .625  and  CT=  .093  (.625)  r  .058 


Heights  at  which  a  given  percent  of  the  specimens  will  fire 
can  be  estimated  from  the  mean  by  subtracting  or  adding 
multiples  of  cT-".  Thus  various  percent  heights  can  be  esti¬ 
mated  from  m±k^by  choosing  the  proper  k. 
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ABSTRACT 


This  report  gives  several  methods  of  estimating  the  mean 
critical  stimulus  and  its  variance. 

The  types  discussed  are: 

1.  Testing  at  single  specification  stimulus. 

2.  Testing  at  a  single  stimulus  greater  than  the 
specification  stimulus. 

3.  Testing  at  two  or  three  stimuli  all  greater 
than  the  specification  stimulus 

4.  A  complete  "run  down"  test  or  probit  testing. 

5.  The  "Staircase"  or  "Bruceton"  method  which 
consists  of  varying  the  stimulus  throughout 
the  test. 
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ADDENDUM 


ACCURACY  OF  CALCULATIONS 


There  appears  to  be  a  great  deal  of  misunderstanding  regarding 
accuracy  of  calculations  and  significant  numbers.  Some  of  the  confusion 
seems  to  stem,  from  a  misunderstanding  concerning  the  meanings  of 
accuracy  and  precision. 

We  deal  primarily  with  two  types  of  numbers,  rational  and 
irrational,  with  the  rational  numbers  being  either  integral  or  fractional. 
Obviously,  since  we  are  dealing  with  measurements,  our  readings  will 
be  inaccurate  after  a  given  point,  due  to  the  limitations  of  our  measuring 
devices.  Because  of  this,  we  must  discard  some  of  our  figures  resulting 
from  computations.  However,  we  may  also  discard  some  figures  if  the 
accuracy  of  our  results  are  only  needed  to  a  certain  degree.  In  addition, 
certain  numbers  such  as  n  ,  e,  roots,  logarithms  can  only  be  approxi¬ 
mated  to  being  irrational  numbers. 

For  the  reasons  given  above,  we  must  express  our  results  as 
"correct  to  the  nth  significant  figure". 

Definitions 

Approximate  number  -  An  approximate  number  is  one  that  differs  from 
the  exact  number  for  which  it  stands.  For  example,  1/3  -  .333;  jt  ~  3. 14; 
546,  273,  812  ~  546,  274,  000.  The  symbol  ~  means  "is  approximately". 

Significant  figure  -  A  significant  figure  is  any  digit  from  1  through  9;  a 
zero  is  significant  except  when  it  is  used  for  the  sole  purpose  of  lixing 
the  decimal  point.  Zeros  at  the  end  of  a  number  such  as  546,  273,  000 
may  or  may  not  be  significant.  Therefore,  a  proper  procedure  to  follow 
is  to  write  numbers  in  a  scientific  form  where  the  number  is  expressed 
as  one  whole  number  with  the  decimals  to  the  proper  number  of  significant 
digits  and  the  magnitude  powers  of  10.  For  example,  5.46  x  10^  is  sig¬ 
nificant  to  three  digits,  but  its  magnitude  is  in  the  ten  thousands. 

The  rules  to  follow  in  rounding  off  a  number  to  n  significant 
figures  are  as  follows: 

a.  Discard  all  digits  to  the  right  of  the  n^  digit. 
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b.  Increase  the  digit  by  one  if  the  n  plus  first  digit  is 
greater  than  5. 

c.  Do  not  change  the  n^  digit  if  the  n  plus  first  digit  is  less 

than  5. 

d.  If  the  n  plus  first  digit  is  exactly  5,  increase  the  nth  digit 
by  1  if  it  is  odd;  let  it  remain  unchanged  if  it  is  even.  For  example, 
in  rounding  off  2.  565  to  three  significant  figures  gives  2.  56  but  2.  555 
would  round  off  to  2.  56. 

Therefore  a  number  which  has  been  rounded  off  to  n  significant 
figures  may  differ  from  the  actual  number  by,  at  most,  one  integer  in 
the  nth  place. 

Measurements  of  Error 

One  measurement  of  error  is  the  difference  between  the  true 
value  of  a  quantity  and  its  approximate  value  and  is  known  as  the  absolute 
error.  For  example,  1  inch  may  be  read  as  1.02  inch.  Here,  the 
absolute  error  is  0.02  inch. 


For  a  number  that  is  correct  to  n  significant  figures,  its  absolute 
error  cannot  be  greater  than  1/2  unit  of  the  nth  significant  figure.  This 
follows  directly  from  the  rounding  off  process.  Thus,  if  2.  568  is  correct 
to  four  significant  figures,  the  true  value  must  lie  in  the  interval  i  -om 
2.5675  to  2.5685. 

Another  measurement  of  error  is  known  as  the  relative  error 
and  is  defined  as  the  ratio  between  the  absolute  error  and  the  true  value. 


Even  though  an  absolute  error  is  greater  in  magnitude,  the  measurement 
may  be  more  accurate  than  one  having  a  smaller  absolute  error.  For 
example,  in  measuring  a  2  inch  block  to  the  nearest  thousandth  of  an 
inch,  and  one  mile  to  the  nearest  foot,  the  absolute  errors  are  0.0005 
inch  and  6  inches  respectively.  However,  the  respective  relation  errors 
are  0. 005  or  _ 1  and  12  or  1  ,  clearly  showing  that 

2  ~ 4000  5280~  10, 560 

the  measurement  of  the  mile  is  more  accurate. 

The  best  estimate  of  error  is  the  standard  deviation  which  is  a 
measure  of  the  dispersion  of  the  readings  about  the  average.  The 
formula  for  the  standard  deviation; 

.  Jzo C,  -  X)2 


1  -  POh 


where  Xj  is  the  individual  readings,  X  is  the  average  of  all  the  readings 
and  n  is  the  number  of  Individual  readings.  However,  if  a  desk  calculator 
is  used  the  formula  may  be  written 


o>i)2 

n(n  -  1) 


’ji  general,  a  number  correct  to  n  significant  figures  will  be  in  the  interval 
between  x  +  3  a  and  x  -  3  a  . 


Quite  often,  it  is  required  to  know  how  many  figures  to  carry  in 
computational  work.  In  adding  a  column  of  not  more  than  20  figures,  one 
more  decimal  place  should  be  carried  than  is  required  in  the  final  result. 

If  a  measurement  is  needed  correct  to  the  nearest  hundredth,  then  the 
individual  readings  are  taken  to  the  nearest  thousandth.  Since  the  absolute 
error  of  each  of  the  numbers  will  be  <  0.0005;  for  twenty  numbers  it  will 
not  exceed  0.01.  Therefore,  it  will  be  possible  for  the  second  decimal 
to  be  in  error  by  not  more  than  one  unit. 


1 

l 


o 

v' 

V' 

it 


A 


If  there  are  more  than  20  but  less  than  200  readings,  carry  two 
more  decimals  than  are  required  in  the  final  result. 

When  subtracting  numbers,  both  should  be  rounded  off  to  the 
same  number  of  decimal  places.  For  example,  in  subtracting  54.  563 
from  764.  9  where  each  is  correct  to  the  total  number  of  significant 
figures.  It  would  be  entirely  wrong  to  write  764.900  -  54.563  because 
the  last  two  places  in  764.  900  are  not  necessarily  zero.  The  accepted 
way  is  to  write  764.  9  -  54.6.  This  type  of  error  is  often  found  in 
writing  of  tolerance  limits,  where  the  true  dimension  is  given  as  0.06 

±  10°c. 


A  serious  error  occurs  in  the  subtraction  of  two  nearly  equal 
approximate  numbers.  In  subtracting  55.563  from  55.436  which  are 
each  correct  to  five  figures  we  obtain  the  result  of  0.  127,  correct  to 
only  three  significant  figures.  In  cases  where  the  numbers  are  ex¬ 
tremely  close,  the  result  may  be  correct  to  only  one  significant  figure, 
and  even  this  may  be  in  error  by  one  unit.  For  this  type  of  computa¬ 
tion,  in  order  to  eliminate  this  occurrence,  we  must  carry  additional 
places.  This  can  be  done  if  the  numbers  are  approximated  such  as 
v  ,  VT,  e,  and  so  forth. 
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Supposing  we  had  a  problem  in  multiplication  where  one  factor 
is  without  error  and  the  other  is  an  approximate  number  and  we  wish 
the  absolute  error  to  be  less  than  some  fixed  constant.  In  a  case  of 
this  sort,  retain  as  many  decimal  places  in  the  approximate  factor 
2=  th°re  are  whole  digits  in  the  other  factor  plus  the  number  of  decimal 
places  permitted  in  the  absolute  error.  Let  us  consider  the  case  of 
multiplying  2'"»1  by'  v  where  291  is  known  without  error.  Let  us  assume 
that  we  wish  the  absolute  error  to  be  <  .01.  To  obtain  this  magnitude 
of  the  absolute  error,  we  retain  5  decimal  places  for  rr  .  If  the  de¬ 
sired  absolute  error  has  been  0.001,  then  6  decimal  places  would 
have  been  retained. 

In  the  case  where  both  factors  are  approximate,  retain  as 
mar.y  decimals  in  the  multiplicand  as  there  are  whole  numbers  in  the 
limiting  error.  In  the  multipler,  retain  as  many  decimals  as  there 
are  whole  numbers  in  the  multiplicand,  plus  the  number  of  decimals 
in  the  limiting  error.  For  example,  in  multiplying  30.87541  by 
6.21832  so  that  the  absolute  error  of  the  product  is  <  .01  take  30.  875 
x  6.2183  and  obtain  191.99. 

In  finding  a  product  of  two  or  more  approximate  numbers  of 
different  accuracies,  the  more  accurate  number  is  rounded  off  so  that 
it  contains  one  more  significant  figure  than  the  leat  accurate  factor. 

The  result  should  be  given  to  as  many  significant  figures  as  are  con¬ 
tained  in  the  least  accurate  factor.  A  more  reliable  procedure  is 
given  later. 

In  division,  as  in  multiplication,  if  one  of  the  numbers  is  more 
accurate,  it  should  be  rounded  off  so  as  to  contain  one  more  significant 
figure  than  the  less  accurate  one.  Again,  the  result  should  be  given 
to  as  many,  but  not  more,  significant  figures  as  are  contained  in  the 
less  accurate.  Dividing  56.  3  by  V5~where  the  numerator  is  an  approxi¬ 
mate  number  which  has  been  rounded  off  to  one  decimal  place.  In 
order  to  minimize  the  error  of  the  V37  we  take  the  value  2.  236  as  the 
square  root  of  5.  Using  the  relative  error,  E~  <  .05  <;  .0009  the 

“  56.3 

division  56.3  _  25.  2  and  Ea<  25.  2  x  .  0009  <.023.  Since  this  error 

27l3F~  = 

has  no  effect  on  the  third  place  of  the  quotient,  we  can  accept  25.  2  as 
the  correct  result. 
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12.39  12.4 

Average  i.  239  -  1.  24  Average  1.  24 

The  accuracy  of  a  product  can  be  found  by  use  of  the  relative 
error.  The  absolute  error  is  then  found  from: 

absolute  error  =  relative 
error  x  product  of  factors 


For  a  series  of  products,  the  relative  error  is: 
„  1  ^  2  AUr i 

Ml  M  2  M  „ 


In  averaging  numbers  where  the  number  being  used  to  obtain  the 
average  >  10,  one  more  place  may  be  considered  significant  than  was 
significant  in  the  terms  of  the  sum.  Seldom  should  more  places  be  re¬ 
tained  in  the  average. 

For  example 


with  N 

equal  to  the  product  of  u  y  ug-  U3 _ un.  Consider  the  product  349. 1  x 

863.  4  with  both  factors  correct  to  four  figures.  Therefore,  Auj  =Au2  =  .( 

and  E~<_JL5. —  — =  .00020. 

349.1  8G3.4 

Here,  N  ~  301413.  The  absolute  error  is  —  301413  x  .00020  =  60. 
The  true  result  must  therefore  lie  between  301473  and  301353.  However, 
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since  from  this  we  see  that  we  can  only  trust  4  significant  figures  we 
write  349.  1  x  863.4  =  3.014  x  105. 


Let  us  consider  the  above  problem  from  the  standpoint  that  the 
true  product  must  be  between  its  maximum  and  minimum  values.  We 
then  retain  as  many  significant  figures  as  coincide  in  the  maximum 
and  minimum  values.  The  maximum  value  is  349.  15  x  863.45  =  3014.74 
and  the  minimum  is  349.05  =  863.35  =  -3013.52.  This  points  out  that 
we  can  retain  four  significant  figures.  The  effect  of  dropping  right 
hand  numbers  in  multiplicand  and  multiplier  is  shown  below: 
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Table  I 


Areas  r>nd  Their  Standard  Sampling  Errors  for  the  Normal 
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2312774 
.2302383 
,2291750 
2280879 
2269775 
2258441 
224  6881 
,2235101 
2223103 
2210893 
2198475 
2185853 
2173032 
2160017 
2146811 
2133421 
2119850 
2106104 
2092186 
2078102 
2063857 
2049456 
2034902 
2020203 
2005361 
1990383 
1975273 
1960037 
1944678 
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Table  I.  Continued 


T 

Q 

PQ 

.  64 

.2610863 

.1929202 

.65 

.2  5784  61 

.1913615 

.66 

.2546269 

.1897920 

.67 

.2514289 

.1882124 

.68 

.2482522 

.1866230 

.69 

.2450971 

.1850245 

.70 

.2419637 

.1834173 

.71 

.2388521 

.1818018 

.72 

.2357625 

.1801785 

.73 

.2326951 

.  17  8  5481 

.74 

.2296500 

.1769109 

.75 

.2266274 

.1752674 

.76 

.2236273 

.1736181 

.77 

.2206499 

.1719635 

.78 

.2176954 

.  1703041 

.79 

.2147639 

.1686404 

.80 

.2118554 

.1669727 

.81 

.2089701 

.1653016 

.82 

.2061081 

.1636276 

.83 

.2032694 

.1619510 

.84 

.2004542 

.1602723 

.85 

.1976625 

.1585920 

.86 

.1948945 

.1569106 

.87 

.1921502 

.1552285 

.88 

.1894297 

.15  3  54  61 

.89 

. 1867329 

. 1518637 

.90 

.1840601 

.1501820 

.91 

.1814113 

.1485012 

.92 

.1787864 

.1468218 

.93 

.1761855 

.  1451442 

.94 

.1736088 

.1434  688 

.95 

.1710561 

.1417959 

.96 

.1685276 

.  1401260 

.97 

.16602032 

.1384595 

.98 

.  163  54  31 

.1367968 

.99 

.1610871 

.1351380 

1.00 

,1580553 

. 1334838 

1.01 

.1562476 

.1318343 

I. 

Q 

PQ 

1.02 

.153  8  642 

.1301900 

1.03 

.1515050 

.1285512 

1.04 

.1491700 

.1269183 

1.05 

.1468591 

.1252915 

1.06 

.1445723 

.1236712 

1.07  - 

.1423097 

.1220576 

1.08 

.1400711 

.1204512 

1.09  - 

.1378566 

.1188522 

1.10 

.1356661 

.1172608 

1.11  - 

.1334995 

.1156774 

1.12 

.1313569 

.1141023 

1.13  - 

.1292381 

.1125356 

1.14 

.1271432 

.1109778 

1.15 

.1250719 

.1094289 

1.16 

.1230244 

. 1078894 

1.17 

.1210005 

.1063594 

1.18 

.1190001 

.1048391 

1.19  -4 

.1170232 

.1033288 

1.20 

.1150697 

.1018287 

1.21 

.1131394 

.1003389 

1.22 

.1112324 

.0988598 

1.23 

.1093486 

.0973915 

1.24 

.1074877 

.0959341 

1.25 

.1056498 

.0944879 

1.26 

.1038347 

.0930531 

1.27 

.1020423 

.0916297 

1.28 

.1002726 

.0902180 

1.29 

.0985253 

.0888181 

1.30 

.0968005 

.0874302 

1.31 

.0950979 

.0860543 

1.32 

.0934175 

.0846907 

1.33 

.0917591 

.0833394 

1.34 

.0901227 

.0820006 

1.35 

.0885080 

.0806743 

1.36 

.0869150 

.0793608 

1.37 

.0853435 

.0780600 

1.38 

.0837933 

.0767720 

1.39 

.0822644 

.0754970 

3 


.■j 
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Table  I.  Continued 


I 


i 

} 


T  Q  PQ 


1.40 

. 0807567 

.0742351 

1.41 

. 0792698 

.0729861 

1.42 

.0778038 

.0717504 

1.43 

.0763585 

.0705279 

1.44 

. 0749337 

.0693186 

1.45 

.0735293 

.0681227 

1.46 

.0721450 

.0669401 

1.47 

. 0707809 

.0657710 

1.48 

.0694366 

.0646  1  52 

1.49 

.0681121 

.0634728 

1.50 

. 0668072 

.0623440 

1.51 

.0655217 

.0612286 

1.52 

.064  2555 

.0601267 

1.53 

. 0630084 

.0590383 

1.54 

.0617802 

.0579634 

1.55 

. 0605708 

.0569020 

1.56 

.0594799 

.0558539 

1.57 

. 0582076 

.0548195 

1.58 

.0570534 

.0537983 

1.59 

.0559174 

.0527906 

1.60 

.0547993 

.0517963 

1.61 

.0536989 

.0508153 

1.62 

.0526161 

.0498476 

1.63 

.0515507 

.04  8  8  9  3  2 

1.64 

.0505026 

.04  7  9  521 

1.65 

.0494715 

.04  7  0241 

1.66 

.0484572 

.0461  091 

1.67 

.0474597 

.0452073 

1.68 

.04  64787 

.0443184 

1.69 

.0455140 

.0434425 

1.70 

.0445655 

.04  2  5  7  94 

1.71 

.0436329 

.04  1  7  2  91 

1.72 

.0427162 

.04  0  8  9  1  5 

1.73 

.0418151 

.04  00666 

1.74 

.0409295 

.0392543 

1.75 

.04  00592 

.0384  54  5 

1.76 

.0392039 

.0376670 

1.77 

. 038363G 

. 0368918 

T_ 

Q 

PQ 

1.78 

.0375380 

.0361289 

1.79 

.0367270 

.0353781 

1.80 

.0359303 

.0346393 

1.81 

.0351479 

.0339125 

1.82 

.0343795 

.0331975 

1.83 

.0336250 

.0324944 

1.84 

.0328841 

.0318027 

1.85 

.0321568 

.0311227 

1.86 

.0314428 

.0304542 

1.87 

.0307419 

.0297968 

1.88 

.0300540 

.0291508 

1.89 

.0293790 

.0285159 

1.90 

.0287166 

.0278920 

1.91 

.0280666 

.0272789 

1.92 

.0274289 

.0266766 

1.93 

.0268034 

.0260850 

1.94 

.0261898 

.0255039 

1.95 

.0255881 

.0249333 

1.96 

.0249979 

.0243730 

1.97 

.0244192 

.0238229 

1.98 

.0238518 

.0232829 

1.99 

.0232955 

.0227528 

2.00 

.0227501 

.0222325 

2.01 

.0222156 

.0217221 

2.02 

.0216917 

.0212212 

2.03 

.0211783 

.0207298 

2.04 

.0206752 

.0202477 

2.05 

.0201822 

.0197749 

2.06 

.0196993 

.0193112 

2.07 

.0192262 

.0188566 

2.08 

.0187628 

.0184108 

2.09 

.0183089 

.0179737 

2.10 

.017  8  644 

.0175453 

2.11 

.0174292 

.0171254 

2.12 

.0170030 

.0167139 

2. 13 

.  0165858 

.0163107 

2.14 

.0161774 

.0159157 

2. 15 

.0157776 

.0155287 

| 

C'-- 


l  -  ?n 


w 


*  ^ 


Table  1.  Continued 


I 


-  I_ 

Q 

PQ 

T_ 

Q 

PQ 

2.16 

.0153863 

.0151496 

2.  57 

.0050849 

.0050590 

2.17 

.0150034 

.0147783 

2.58 

.00494  00 

.0049156 

2.18 

.0146287 

.0144147 

2.  59 

.0047988 

.0047758 

2.19 

.0142621 

.0140587 

2.60 

.0046612 

.004  6395 

2.20 

.0139034 

.0137101 

2.61 

.0045271 

.0045066 

2.21 

.0135526 

.0133689 

2.62 

.0043965 

,0043772 

2.22 

.0132094 

.0130349 

2.63 

.0042692 

.004  2510 

2.23 

.0128737 

.0127080 

2.64  *' 

.0041453 

.0041281 

2.24 

.01  254  5  5 

.0123881 

2.65 

.0040246 

.0040084 

2.25 

.0122245 

.0120751 

2.66  ~ 

.0039070 

.0030917 

2.26 

.0119106 

.0117687 

2.  67 

.  0037926 

. 0037782 

2.27 

.0116038 

.0114692 

2.  68 

.0036811 

.0036675 

2.28 

.0113038 

.0111760 

2.69 

.0035726 

.0035598 

2.29 

.0110107 

.0108895 

2.70 

.0034670 

.0034550 

2.30 

.0107241 

.0106091 

2.71 

.0033642 

.0033529 

2.31 

.0104441 

.0103350 

2.72 

.0032641 

.0032534 

2.32 

.0101704 

.0100670 

2.73 

.0031667 

.0031567 

2.33 

.0099031 

. 0098050 

2.74 

.0030720 

.0030626 

2.34 

.0096419 

.0095489 

2.75 

. 0029708 

.0029709 

2.35 

.0093867 

.0092986 

2.76 

. 0028901 

.0028817 

2.36 

.0091375 

.0090540 

2.77 

. 0028028 

.0027949 

2.37 

.0088940 

.0088149 

2.78 

.0027179 

.0027105 

2.38 

.0086563 

.0085814 

2.79 

.0026354 

. 0026285 

2.39 

.0084242 

.0083532 

2.80 

.0025551 

.0025486 

2.40 

.0081975 

.0081303 

2.81 

.0024771 

.0024710 

2.41 

.0079763 

.0079127 

2.82 

.0024012 

.0023954 

2.42 

.0077603 

.0077001 

2.83 

. 0023274 

.0023220 

2.43 

.0075494 

.0074924 

2.84 

.0022557 

.0022506 

2.44 

. 0073436 

.0072897 

2.85 

.0021860 

.0021812 

2.45 

.0071428 

.0070918 

2.86 

.0021182 

.0021137 

2.46 

. 00694  69 

.0068986 

2.87 

.0020524 

.0020482 

2  47 

.0067557 

.0067101 

2.88 

.0019884 

.0019844 

2.48 

.0065691 

.0065259 

2.89 

.0019262 

.0019225 

2.49 

. 0063872 

.00634  64 

2.90 

.0018658 

.0018623 

2.  50 

.0062097 

. 0061711 

2.91 

.0018071 

.0018038 

2.  51 

.0060366 

.0060002 

2.92 

.0017502 

.0017471 

2.  52 

. 0058G77 

.0058333 

2.93 

.0016948 

.0016919 

2.  53 

.0057031 

.0056706 

2.94 

.0016411 

.0016384 

2.  54 

. 0055426 

.0055119 

2.95 

.0015889 

.0015864 

2.  55 

.  00538  C 1 

.0053571 

2.96 

.0015382 

.0015358 

2.  56 

. 005233G 

.0052062 

2.  97 

.0014890 

.0014868 
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TABLE  I  Continued 


:*\i 


: 

! 


I 


f 


I 

4 


P 

P 

P 

P 


l 


r 


i 


t 


i 

r 

* 


T 

Q 

PQ 

T 

Q 

PQ 

2.  do 

.0014412 

.0014391 

3.36 

.0003897 

.0003895 

2.99 

. 0013949 

.0013930 

3.37 

.0003758 

.0003757 

3.00 

.0013449 

.0013481 

3.38 

.0003624 

. 0003623 

3.01 

.0013062 

.0013045 

3.39 

.0003495 

.0003494 

3.02 

.0012639 

.0012623 

3.40 

.0003369 

. 0003368 

3.03 

.0012228 

.0012213 

3.41 

.0003248 

. 0003247 

3.04 

.0011829 

.0011815 

3.42 

.0003131 

.0003130 

3.05 

.0011442 

.0011429 

3.43 

.0003018 

.0003017 

3.06 

.0011067 

.0011055 

3.44 

.0002909 

.0002908 

3.07 

.0010703 

.0010692 

3.45 

.0002803 

.0002802 

3.08 

.0010350 

.0010339 

3.46 

.0002701 

.0002700 

3.09 

.0010008 

.0009998 

3.47 

.0002602 

.0002601 

3. 10 

.0009676 

.0009667 

3.48 

.0002507 

.0002506 

3.11 

.0009354 

.0009354 

3.49 

..  0002415 

.0002414 

3.12 

.0009043 

.0009035 

3.50 

.0002326 

.0002325 

3. 13 

.0008740 

.00087  3  2 

3.51 

.0002241 

. 0002240 

3.14 

.0008447 

.0008440 

3.52 

.0002158 

.0002158 

3.15 

.0008164 

.0008157 

3.53 

.0002078 

.0002078 

3.16 

.0007888 

.0007882 

3.54 

. 0002001 

.0002001 

3.17 

.0007622 

.0007616 

3.55 

.0001926 

.0001926 

3.18 

.0007364 

.0007359 

3.56 

.0001854 

.0001854 

3. 19 

.0007114 

.0007109 

3.57 

.0001785 

.0001785 

3.20 

.0006871 

.0006866 

3.58 

.0001718 

.0001718 

3.21 

.0006637 

.0006633 

3.59 

.  .0001653 

.0001653 

3.22 

.0006410 

.0006406 

3.60 

.0001591 

.0001591 

3.23 

.0006190 

.0006186 

3.61 

.  .00^1531 

.0001531 

3.  24 

.0005976 

.0005972 

3.62 

.0001473 

.0001473 

3.25 

.0005770 

.0005767 

3.63 

.  .0001417 

.0001417 

3.  26 

.0005571 

.0005568 

3.64 

.0001363 

.0001363 

3.27 

.0005377 

.0005374 

3.  65 

.  .0001311 

.0001311 

3.28 

.0005190 

.0005187 

3.  66 

.0001261 

.0001261 

3.29 

.0005009 

.0005006 

3.67 

_  .0001213 

.0001213 

3.30 

.0004834 

.0004832 

3.  68 

.0001166 

.0001166 

3.31 

.0004665 

.0004663 

3.  69 

.0001121 

.0001121 

3.32 

.0004501 

.0004499 

3.70 

.0001078 

.0001078 

3.33 

.0004342 

.0004340 

3.71 

.0001036 

.0001036 

3.34 

.0004189 

.0004187 

3.72 

.0000996 

.0000996 

3.35 

.0004041 

.0004039 

3.73 

.0000957 

.0000957 

K 
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TABLE  I  Continued 


T 

_Q_ 

PQ 

3.74 

. 0000920 

.0000920 

3.75 

. 0000884 

.0000884 

3.76 

. 0000850 

.0000850 

3.  77 

.0000816 

.0000816 

3.78 

.0000784 

.00007  84 

3.79 

.0000753 

.0000753 

C.  80 

.0000723 

.0000723 

3.81 

.0000695 

.0000695 

3.  82 

. 0000667 

.0000667 

3  83 

. 0000641 

.0000641 

3.84 

.0000615 

.0000615 

3.  85 

.0000591 

.0000591 

3.  86 

.0000567 

.0000567 

3.  87 

.0000544 

. 0000544 

3.  88 

.0000522 

. 0000522 

3.89 

.0000501 

.0000501 

3.90 

.0000481 

. 0000481 

3.91 

.  0000461 

. 0000461 

3.92 

.0000443 

.0000443 

3.93 

.0000425 

. 0000425 

3.94 

. 0000407 

.0000407 

3.95 

.0000391 

.0000391 

3.96 

.0000375 

.0000375 

3.97 

.0000359 

.0000359 

3.98 

.0000345 

. 0000345 

3.99 

.0000330 

. 0000330 

4.00 

.0000317 

.0000317 

4.01 

. 0000304 

.0000304 

4.  02 

.0000291 

.0000291 

4.03 

.0000279 

.0000279 

4.04 

.0000267 

.0000267 

4.05 

.0000256 

.0000256 

4.06 

.0000245 

.0000245 

4.07 

.0000235 

.0000235 

4.08 

.0000225 

.0000225 

4.09 

.0000216 

.0000216 

4. 10 

.0000207 

.0000207 

4. 11 

.0000198 

.0000108 

T 

_Q_ 

PQ 

4.12 

.0000189 

.0000189 

4.13 

.0000181 

.0000181 

4.14 

. 0000174 

.0000174 

4.15 

.0000166 

.0000166 

4.16 

.0000159 

.0000159 

4. 17 

.0000152 

.0000152 

4.18 

.0000146 

.0000146 

4.19 

.0000139 

.0000139 

4.20 

.0000133 

.0000133 

4.21 

.0000128 

.0000128 

4.22 

.0000122 

.0000122 

4.23 

.0000117 

.0000117 

4.24 

.0000112 

.0000112 

4.25 

.0000107 

.0000107 

4.26 

.0000102 

.0000102 

4.27 

.0000098 

.0000098 

4.28 

.0000093 

.0000093 

4.29 

.0000089 

.0000089 

4.30 

.0000085 

.0000085 

4.31 

.0000082 

.0000082 

4.32 

.0000078 

.0000078 

4.33 

.0000075 

.0000075 

4.34 

.0000071 

.0000071 

4.  35 

.0000068 

.0000068 

4.  36 

.0000065 

.0000065 

4.  37 

. 0000062 

. 0000062 

4.38 

.0000059 

.0000059 

4.39 

. 0000057 

. 0000057 

4.  40 

.0000054 

.0000054 

4.41 

.0000052 

.0000052 

4.42 

.0000049 

.0000049 

4.43 

. 0000047 

. 0000047 

4.44 

. 0000045 

.0000045 

4.45 

.0000043 

.0000043 

4.46 

.0000041 

.0000041 

4.  47 

.0000039 

.0000039 

4.48 

.0000037 

. 0000037 

4.49 

.0000036 

.0000036 

l  -  ?U 
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Q 

PQ 

. 0000034 

. 0000034 

. 0000032 

. 0000032 

. 0000031 

. 0000031 

. 0000030 

. 0000030 

. 0000028 

. 0000028 

. 0000027 

. 0000027 

. 0000026 

.0000026 

. 0000024 

.0000024 

. 0000023 

. 0000023 

. 0000022 

. 0000022 

.0000021 

.  0000021 

. 0000020 

. 0000020 

. 0000019 

. 0000019 

.0000018 

.0000018 

.0000017 

. 0000017 

. 0000017 

.1)000017 

.0000016 

.0000016 

. 0000015 

.0000015 

Table  n 


( 

§ 

% 

i 

i 

i 

» 

I 

\ 

% 


ft 

l 


3 

5- 


N 

* 

§» 


5 

I 


» 


■ 

c* 

»,* 


Ordinates,  Problts,  and  ProbitWeights  Associated  with  the  Normal 
Curve  of  Error 


-T 


2/2 


Z  =  7=?  e 
V2 tr 


W  = 


PQ 


T 

7^ 

W 

.00 

.3989423 

.63662 

.01 

.3989223 

. 63660 

.02 

.3988625 

.63653 

.03 

.3987628 

.63  641 

.04 

.3986233 

.63625 

.05 

.3984439 

.63604 

.06 

.3982248 

.63579 

.07 

.3979661 

.63  549 

.08 

. 3976677 

.63514 

.09 

.3973298 

.63475 

.10 

.3969525 

.63431 

.11 

.3965360 

.63383 

.12 

.3960802 

.63330 

.13 

.3955854 

.63272 

.14 

.3950517 

.63210 

.15 

. 3944793 

.63143 

.16 

.3938684 

.63072 

.17 

.3932190 

.62996 

.18 

.3925315 

. 62916 

.19 

.3918060 

.62832 

.20 

.3910427 

.62742 

.21 

.3902419 

.  62  04  9 

.22 

.3894  038 

.62551 

.23 

.3885286 

.62448 

.24 

.3876166 

. 62341 

.25 

.3866681 

. 62230 

.26 

.3856834 

. 62115 

.27 

.3846627 

.61995 

.28 

.3836063 

.61870 

.29 

.3825146 

.61742 

.30 

' .3813878 

.61609 

T_ 

W 

.31 

.3802264 

.61472 

.32 

.3790305 

.61331 

.33 

. 3778007 

.61185 

.34 

.3765372 

.61035 

.35 

.3752403 

. 60882 

.36 

.3739106 

.60724 

.37 

.3725483 

. 60562 

.38 

.3711539 

.60396 

.39 

. 3697277 

. 60226 

.40 

.3682701 

. 60052 

.41 

.3667817 

. 59874 

.42 

.3652627 

. 59692 

.43 

.3637136 

. 59506 

.44 

.3621349 

.59316 

.45 

.3605270 

.59123 

.46 

.3588903 

.58925 

.47 

.3572253 

.  58724 

.48 

.3555325 

.58520 

.49 

.3538124 

.58311 

.50 

.3520653 

. 58099 

.51 

.3502919 

. 57884 

.  52 

.3484925 

.57664 

.53 

.3466677 

.57442 

.54 

.3448180 

.57215 

.  55 

.3429439 

.56986 

.  56 

.3410458 

.56753 

.  57 

.3391243 

.56516 

.58 

.3371799 
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.59 

,3352132 
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.60 

.3332246 

. 55788 

.61 
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.55538 
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.76 
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1.17 
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1.18 
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.1965205 

.79 
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1.21 
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1.23 
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1.26 
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.88 
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.96 
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.97 
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1.41 

. 1476385 

1.01 

.2395511 

.43528 
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.02923 

2.  35 

.0252182 

.06839 
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.05915 
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.05315 

2.86 

.0066793 

.02111 

2.48 

.0184233 

.05201 

2.87 

.0064907 

,02057 

2.49 

.0179711 

.05089 

2.88 

. 0063067 

. 02004 

2.50 

.0175283 

.04979 

2.89 

.0061274 

.01953 

2.51 

.0170947 

.04870 

2.90 

.0059525 

.01903 

2.52 

.0  1  66701 

.04764 

2.91 

.0057821 

.01853 

2.53 

.0162545 

.04659 

2.92 

.0056160 

.01805 

2.  54 

.0158476 

.04556 

2.93 

.0054541 

.01758 

2.55 

.0154493 

.04455 

2.94 

.0052963 

.01712 

2.  56 

.0150596 

.04356 

2.95 

.0051426 

.01667 

2.57 

.0146782 

.04259 

2.96 

. 0049929 

.01623 

2.  58 

.0143051 
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.0044318 

.01457 

2.62 

.0128921 
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FIGURE  7.  GRAPH  FOR  FIRST  APPROXIMATION 


BULLPUP,  A  ROUND  OF  AMMUNITION 

R.  D.  GINTER,  Senior  BULLPUP  Project  Engineer,  Bureau  of  Naval  Weapons 


For  those  of  you  who  are  not  familiar  with  BULLPUP,  or  who  might  be 
thinking  that  the  Bureau  is  shooting  dogs  into  space,  this  is  it.  A  570 
pound,  air-to-surface  missile,  designed  to  provide  effective  close  air 
support  and  ground  interdiction  capability  to  light  attack  aircraft. 

Visually  guided  by  a  radio  comnand  link,  it  is  the  only  missile  in  the 
world  which  is  never  tested  after  it  leaves  the  contractor’s  plant. 

BULLPUP  is  an  example  of  the  tangible  benefits  which  can  be  achieved 
in  operational  readiness  and  system  cost  reduction  with  high  reliability. 

A  reliability  which  is  sufficiently  high  for  the  Navy  to  firmly  commit 
the  EULLPUP  system  to  operate  without  the  use  of  any  missile  test  equip¬ 
ment.  All  BULLPUP  missiles  are  "up”  until  actual  flight  proves  differently. 
They  are  treated  as  a  "round  of  a/nnunition".  Under  this  "no-test-equipment" 
philosophy,  BUILPUP  has  exceeded  both  specification  requirements  and  our 
own  optimistic  expectations.  Reliability  in  the  Fleet  is  in  excess  of 
90*. 


It  would  be  fitting  if  I  could  reveal  a  secret  formula  or  magic  pro¬ 
cedure  for  insuring  reliability  in  electronic-missile  production. 
Unfortunately,  BULLPUP  hat  not  discovered  such  a  formula.  Primarily, 
we  have  applied  time-worn  cliches,  paid  attention  to  the  smallest  detail 
and  been  adamant  about  stringent  control  of  all  factors  comprising  the 
entire  system. 

BULLPUP  specifications  have  always  required  that  the  missile  meet  a 
particular  reliability  goal.  This  goal  has  been  continually  increased 
during  development,  evaluation  and  now  production.  All  of  these  specifi¬ 
cations  recognized  that  continued  testing  of  the  missile  during  "ready 
service"  stowage  would  reduce  the  total  missile  reliability.  The  speci¬ 
fications  allowed  a  reduction  in  reliability  after  each  test.  That 
reliability  would  be  reduced  due  to  continued  testing  was  proven  during 
the  Nav/  Technical  Evaluation.  A  block  of  JO  missiles  was  subjected  to 
repeated  operational  tests.  None  failed  on  the  first  or  second  tests. 
Failures  began  to  appear  on  the  third  and  subsequent  tests.  Testing 
missiles  in  an  attempt  to  prove  them  ready  was  actually  causing  them  to 
fail. 
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For  a  missile  like  BULLPUP,  it  is  impossible  to  design  test  equipment 
that  will  actually  test  all  of  the  missile.  Items  auch  as  the  motor, 
fuze,  warhead  and  flares,  are  not  amenable  to -test.  These  items  comprise 
fully  two  thirds  of  the  total  missile  volume.  Thus,  regardless  of  the 
quantity  of  the  test  equipment,  the  entire  missile  can  never  be  tested. 

In  addition,  specific  design  improvements  increased  rather  than  decreased 
the  number  of  items  which  cannot  be  tested.  An  example  of  this  in  BULLPUP 
was  the  change  from  an  electric  gyro  to  a  "one^shot  **  spring-wound  gyro. 

The  batteries  and  pneumatic  accumulator  were  also  designed  as  "one-shot** 
items.  In  fact,  the  only  major  item  remaining -which  could  be  tested  was 
the  receiver.  The  problem  finally  became  one  of  insuring  that  the  receiver 
reliability  would  be  adequate  for  the  intended  usage  of  the  missile. 

The  decision  was  made  that  instead  of  re-designing  the  test  equipment 
for  test  of  the  receiver  only,  we  would  eliminate  the  test  equipment. 

The  BULLPUP  Operational  Evaluation  was  started  without  missile  test  equip¬ 
ment  in  early  1958.  Portions  of  this  evaluation  were  highly  successful 
and  other  portions  were  not  so  successful.  Approximately  half  way  through 
the  evaluation,  the  reliability  dropped  from  an  exceedingly  high  level  to 
a  very  low  level.  The  causes  for  this  drop  were  eventually  determined  and 
remedied.  It  is  significant  that  during  all  of  the  review  to  discover  the 
cause  for  the  reliability  drop,  only  one  discrepancy  was  ever  discovered 
which  would  have  been  detected  by  the  missile  test  equipment.  So  even  our 
most  serious  reliability  problem  served  to  strengthen  our  conviction  that 
the  "no-test-equipment"  philosophy  was  eound. 

During  all  of  the  investigations  associated  with  the  drop  in  reli¬ 
ability,  the  extreme  importance  of  stringent,  detailed  control  became 
painfully  evident.  Each  discrepancy  which  was  discovered  could  have  been 
prevented,  if  the  proper  controls  had  originally  been  applied.  We  were 
experiencing  a  number  of  the  problems  usually  associated  with  increasing 
the  production  rate  from  a  few  per  month  to  hundreds  per  month.  Control 
became  the  single  most  important  word  in  the  BULLPUP  vocabularly: 

Control  of  Change 
Control  of  Specifications 
Control  of  Inspection  Procedures 
Control  of  Manufacturing  Techniques 
Control  of  Technical  Improvements 

We  found  that  no  deviation  whatsoever  could  be  allowed  from  the  established 
minimum.  If  a  minimum  standard  had  not  been  set  for  a  particular  item, 
it  had  to  be,  immediately.  The  existance  of  an  actual  standard  from  which 
to  measure  had  to  be  insured.  In  the  event  the  standard  was  either  too 
high  or  too  low,  this  could  be  realistically  modified  later.  The  item  of 
crucial  importance  was  that  no  unit  was  ever  passed  through  inspection 
which  did  not  meet  the  established  minimum. 
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These  areas  of  detailed  control  are  very  standard  and  the  normal 
reaction  ia  that  every  competent  manufacturer  is  thoroughly  aware  of  them. 

The  present  BULLPUP  philosophy  is:  That  we  are  from  Missouri.  We  will 
believe  that  any  manufacturer  has  this  detailed  control  after  it  has  been 
demonstrated,  but  not  before. 

The  control  item  which  gave  us  the  most  difficulty  and  still  does  for 
that  matter,  is  to  insure  that  decisions  are  made  at  the  proper  level  in 
both  the  contractor’s  organization  and  in  the  Government.  Organizational 
control  of  decisions  involves  all  of  the  other  controls.  It  is  mandatory 
that  the  total  organization  responsible  for  the  delivery  of  a  highly 
reliable  missile  to  the  Fleet  be  thoroughly  aware  of  the  objective  for 
that  missile.  This  objective  is  basically  reflected  in  the  specifications 
and  test  procedures.  For  the  working  level  engineers,  inspectors,  etc., 
tnere  is  no  such  thing  as  acceptable  unless  it  is  above  the  specification 
or  test  minimum.  Anything  lower  than  the  established  minimum  must  be 
automatically  unacceptable.  We  have  found  that  human  beings  are  very 
prone  to  place  their  own  intrepretation  on  what  should  or  should  not  be 
accepted  based  on  their  own  individual  intrepretation  of  what  was  needed 
for  the  program.  This  intrepretation  appears  to  vary  as  the  pressure  to 
meet  delivery  deadlines  varies.  Organizational  control  of  these  decisions 
is  one  of  the  very  important  factors  in  insuring  that  a  uniform  production 
item  is  being  delivered.  I  have  been  leading  up  to  the  word  "uniform”. 
Statistics  are  based  on  homogenous  populations.  If  the  population  is  not 
homogenous,  the  numbers  derived  from  the  best  statisticians  are  merely 
numbers.  They  are  not  meaningful.  The  numbers  become  meaningful  and 
capable  of  influencing  intelligent  decisions  when  the  knowledge  is  definite 
and  irrefutable  that  all  units  of  production  under  consideration  are  in 
fact  all  above  the  established  minimum.  Without  the  knowledge  that  the 
statistical  tests  results  are  based  on  uniform  production,  decisions  will 
probably  be  in  error. 

Still  another  area  of  control  which  is  equally  important  is  "Control 
of  Technical  Improvement".  As  you  all  know,  any  item  can  be  improved  to 
death.  Possibly  one  of  the  hardest  Jobs  in  the  management  of  any  program 
i3  to  actually  stick  to  the  decision  that  the  product  now  being  delivered 
is  "good  enough".  At  some  point  in  any  program,  it  becomes  mandatory  that 
the  program  management  stipulate  that  the  item  is  now  GOOD.  Additional 
engineering  improvements  will  not  be  considered  unless  they  are  in  fact 
significant  improvements  to  the  total  system.  In  BULLPUP,  we  will  not 
accept  any  improvement  change  which  would  cause  us  to  depart  from  the  "no¬ 
test-equipment"  philosophy.  We  literally  can’t  afford  to.  We  can  use  a 
number  of  words  to  describe  this  phase  of  a  program:  Design  Freeze,  Final 
Optimization,  etc.  In  BULLPUP,  we  have  chosen  to  call  it  "good  enough". 

The  Navy  has  said  that  the  BUJLPUP  missile  in  its  present  configuration  is 
"good  enough".  It  meets  the  original  performance  specifications  established 
for  it,  plus  a  satisfactory  margin.  The  problem  now  is  to  deliver  Fleet 
usable  units  which  meet  the  minimum  standard  of  "good  enough"  with  a  high 
degree  of  reliability. 
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We  know  that  BULLPUP  will  do  its  job.  We  know  that  it  will  do  this 
job  with  a  high  degree  of  reliability.  We  know  that  the  elimination  of 
test  equipment  from  the  system  has  resulted  in  very  substantial  savings 
to  the  Government.  Thera  are  no  DOWN  BULLPUP  missiles,  no  missile  main¬ 
tenance  problems,  no  test  equipment,  no  maintenance  training  problems,  no 
shortages  of  spares,  etc.  This  list  of  NO* s  has  been  officially  trans¬ 
lated  into  dollar  savings  for  the  first  year’s  outfitting  as  follows: 

Missile  Test  Equipment 
Test  Equipment  Facilities 
Training  and  Personnel  Support 
Spares 

TOTAL  SAVING  $6,767,000 

a.  Missile  teet  equipment;  65  pieces  at  $45,000  each  for  a  total  of 

$3,000,000. 

b.  The  facilities  to  house  the  equipment  at  Naval  Ammunition  Depots, 
Air  Stations,  Overhaul  and  Repair  Stations  and  aboard  three  carriers  was 
estimated  at  $1,000,000.  Due  to  the  space  problem  aboard  the  carriers, 
this  item  was  both  costly  and  time  consuming. 

c.  Training  costs  including  personnel  maintenance  and  support  was 
actually  estimated  at  $3,000,000.  However,  to  insure  a  conservative 
tabulation  of  the  cost  eavings,  this  item  was  listed  as  only  $1,000,000. 

d.  The  amount  of  spare  parts  required  to  support  the  first  year’s 
production  program  was  drastically  reduced.  Prior  to  the  implementation 
of  the  "no-test-equipment"  philosophy,  1.8  million  dollars  had  been  ear¬ 
marked  for  spares.  Due  to  the  "no-test-equipment"  philosophy,  this  order 
was  reduced  to  merely  $38,000.  Both  amounts  were  derived  using  the  same 
basic  provisioning  techniques.  As  a  matter  of  interest,  this  particular 
saving  was  used  to  significantly  increase  the  follow-on  procurement  order 
of  missiles.  Thus,  several  hundred  additional  missiles  were  procured 
instead  of  spare  parts.  It  was  not  possible  to  put  all  of  the  savings 
into  additional  missiles.  However,  the  total  $6,767,000  does  represent 

a  large  amount  of  money  released  from  spares  support,  test  equipment,  etc. 
which  can  be  applied  to  the  funding  of  other  operational  requirements. 

1  have  been  adamant  about  the  necessity  for  a  stringent  control.  We 
look  toward  the  day  when  some  of  these  extremes  might  be  relaxed.  At 
present,  I  cannot  predict  when  this  might  occur.  It  will  definitely  not 
occur  until  a  frozen  design  is  in  production  at  a  stabilized  rate.  We 
are  now  in  the  process  of  building  the  production  rate  for  BULLPUP  to  the 
highest  in  the  missile  history.  After  this  has  been  successfully  accom¬ 
plished,  we  will  face  the  problem  of  control  relaxation.  Speaking  as  a 
Government  representative,  I  will  indicate  that  unless  this  relaxation  of 
control  is  accomplished  with  a  reasonable  decrease  in  unit  coBt,  there 
would  be  little  incentive  for  any  Government  agency  to  relax  control  for 


the  mere  sake  of  relaxation. 


BULLPUP  has  demonstrated  the  magnitude  of  tangible  rewards  obtainable 
through  high  reliability.  We  have  learned  again  the  extreme  importance 
or  strict,  intelligent  control.  Fleet  readiness  has  been  increased  at  a 
significantly  reduced  cost.  All  operational  aspects  of  the  system  have 
been  simplified.  We  believe  that  other  production  missile  systems  can 
profitably  adopt  at  least  portions  of  the  "no-test-equipment"/,,round  of 
rmmunition"  concept. 

This  sounds  like  Pm  advocating  something  as  impossible  as  being  a 
"little  bit  pregnant".  In  this  instance,  1  probably  am.  I’m  advocating 
a  different  approach  to  the  test  equipment  problem.  During  the  definition 
of  the  operational  missile  system,  start  at  the  point  of  "no-test-equip¬ 
ment."  and  work  back.  Make  each  item  of  equipment  Justify  its  existence 
from  the  standpoint  of  total  system  operation  cost.  Allow  nothing  that 
is  merely  nice  to  have  or  easy  to  provide.  Review  the  development  test 
results  and  determine  which  equipment  or  portions  of  the  equipment  actually 
served  a  useful  purpose.  Do  not  allow  the  mere  finding  of  a  fault  by  the 
equipment  to  Justify  the  item.  Instead,  make  every  conceivable  effort  to 
remedy  the  fault  in  design  or  in  control  of  design  on  the  production  line, 
thereby  eliminating  the  test  equipment  item. 

We  believe  that  if  an  approach  similar  to  this  is  applied  to  missile 
systems  producing  large  quantities  of  hardware,  a  very  large  amount  of 
the  present  test  equipment  can  be  eliminated.  Hense,  probably  you,  too, 
are  capable  of  being  at  least  a  "little  bit  pregnant".  If  not  actually 
pregnant,  at  least  you  will  be  "well  loved"  for  a  tremendous  increase  in 
operational  capability. 
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VERIFICATION  OF  FLIGHT  RELIABILITY 
MISSILE  GUIDANCE  UNIT 


L.  N.  St.  James,  Bell  Telephone  Laboratories,  Inc. 


The  determination  of  the  probability  of  Buccess 
of  a  missile  guidance  unit  can  be  approached  in  a  number  of 
ways.  Perhaps  the  most  direct  and,  from  many  standpoints  a 
very  appealing  method,  is  to  fly  missiles  and  keep  score  of 
the  results.  Figure  1  has  been  taken  from  standard  10$ 
consumer's  risk  sampling  tables  and  can  be  used  to  determine 
the  number  of  missiles  that  must  be  flown  and  the  correspond¬ 
ing  number  that  can  fail.  If  99$  probability  of  success  is 
required,  for  instance,  230  missiles  must  be  flown  with  zero 
failures,  390  with  not  more  than  one  failure,  etc.  This  type 
of  table  provides  a  confidence  of  90$  that  more  than  the 
indicated  number  of  failures  will  be  exceeded  if  the  desired 
level  of  reliability  is  not  attained.  The  procedure  obviously 
Involves  sampling,  so  that  the  restrictions  normally  asso¬ 
ciated  with  a  sampling  process  also  apply.  The  sample  must 
be  randomly  selected  from,  or  otherwise  justified  as  repre¬ 
sentative  of,  a  homogeneous  population  which  can  be  looked 
upon  as  very  large. 

Such  a  test  program  for  a  large  missile  would  cost 
many  millions.  It  could  not  be  justified  before  the  design 
had  been  proved  in  and  it  is  not  feasible  to  U3e  this  type 
of  program  to  prove  in  a  design  because  the  sample  itself 
would  not  be  homogeneous  even  if  some  would  authorize  build¬ 
ing  it.  The  conclusion  is  Inescapable:  Proving  by  flight 
test  alone  that  a  guidance  unit  has  attained  a  reliability 
objective  of  .99  Just  cannot  be  done  when  it  will  do  any  good. 

Why  is  .99  a  good  level  to  choose  rather  than  .90 
or  .95/  which  are  still  high?  Two  or  three  years  ago,  .95 
was  a  very  good  level.  Figure  2  shows  the  unconditional 
engagement  reliability  of  a  battery  in  terms  of  the  number 
of  targets  engaged  at  any  rate  up  to  its  maximum  fire  rate 
and  at  any  time.  This  system  is  so  designed  that  it  can 
fire  at  least  one  more  missile  at  any  target  in  the  event 
that  the  first  falls.  There  are  21  missiles  available  and 
the  unconditional  probability  of  success  falls  off  very 
little  starting  from  .96  up  to  IT  targets.  After  that,  it 
falls  off  very  rapidly.  This  system  assumes  a  .95  missile 
and  the  reason  for  Its  high  engagement  reliability  is  that 
it  can  engage  any  target  at  least  twice  if  necessary.  Re¬ 
move  this  capability  of  a  second  engagement  and  the  lower 
line  results.  There  is  no  way  of  escaping  the  basic  law 
that  missiles,  each  with  a  probability  of  success  of  .95 
engaging  17  targets,  have  a  probability  of  getting  them  all 
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of  .95  or  .4.2.  This,  when  added  to  the  probability  of 
success  of  the  ground  equipment,  results  in  the  lower  line. 

Figure  3  gives  a  diagramic  version  of  the  reli¬ 
ability  function  as  it  relates  to  the  evolution  of  a  system. 
Intended  Function,  at  the  top,  embraces^  this  concept.  It 
also  includes  kind  of  targets,  range,  speed,  environment  of 
use,  etc.  Reliability  enters  at  this  point  as  a  major  de¬ 
sign  parameter  before  the  system  configuration  can  be 
sketched  and  the  design  objectives  of  the  subsystems  can  be 
formulated.  Referring  back  to  the  previous  figure,  the  .92 
capability  could  be  obtained  from  a  system  that  did  not 
have  a  second  shot  capability.  All  that  would  be  necessary, 
on  paper  anyway,  would  be  to  add  a  second  system,  use  twice 
as  many  men  and  fire  nearly  twice  as  many  missiles;  hardly 
an  economic  solution.  A  .7  missile  could  be  used  at  the  ex¬ 
pense  of  multiplying  everything  by  U  instead  of  by  2. 

The  Intended  Function,  then,  requires  a  complete 
specification  of  the  engagement  and  the  probability  of  suc¬ 
cess  required.  If  the  economy  of  this  country  is  to  be  kept 
intact,  the  design  objective  for  such  a  relatively  small  and 
simple  thing  as  a  missile  guidance  unit  must  be  in  the  .99 
ange. 


It  will  be  assumed  at  this  point  that  the  system 
configuration  has  been  tentatively  set  and  the  design  ob¬ 
jective  for  the  guidance  unit  has  been  derived  including  the 
required  reliability.  If  the  design  objective  for  the  guid¬ 
ance  unit  has  been,  in  fact,  adequately  and  completely 
specified  at  this  point,  those  responsible  for  its  design 
can  forget  the  system  concept.  It  is  a  relatively  straight¬ 
forward  process  of  translating  the  design  objective  into 
detail  design  parameters  and' verifiable  test  requirements , 
in  other  words,  Product  Design.  Shortly,  models  become 
available  and  they  should  be  evaluated  to  determine  their 
capability.  Everyone  does  this.  It  is  essential  to  know  if 
it  can  be  made  to  wo~k  at  all.  The  thing  frequently  over¬ 
looked  at  this  point,  however,  is  an  adequate  determination 
of  the  reliability  inherent  in  the  design.  As  has  already 
been  shown,  this  cannot  be  done  in  flight;  there  are  never 
enough  of  them  and  certainly  there  is  no  such  think  as  a 
homogeneous  population  at  this  time.  Assuming  for  the  mo¬ 
ment  that  it  has  been  determined  that  the  reliability 
inherent  in  the  design  is  adequate,  the  information  obtained 
provides  the  basis  for  a  production  evaluation  test  to  be 
applied  to  a  sample  selected  at  regular  intervals  throughout 
the  production  period.  This  is  necessary  in  order  to  secure 
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assurance  that  the  production  process  realizes  the-full 
inherent  design  capability.  Actually,  our  approach  to  both 
of  these  evaluation  phases  is  very  similar  and  based  upon 
the  same  basic  assumptions  so  it  will  only  be  necessary  to 
cover  one  in  detail. 

The  process  found  to  be  adequate  is  shown  in 
Figure  4.  In  reference  to  the  first  item,  the  environments 
of  shipment,  handling  and  storage,  there  exists  an  abundance 
of  information  and  what  is  even  more  important,  they  are 
relatively  easy  to  meet.  The  second  item,  flight  environ¬ 
ment,  is  a  real  problem.  Telemetry  has  been  used  for  years 
but  it  has  a  limited  band  width  and  dynamic  range  and  it3 
reliability  has  never  been  particularly  high.  Far  greater 
success  can  be  expected  from  the  flyable  tape  recorder  sys¬ 
tem  shown  on  Figure  5«  In  this  system  there  are  12  avail¬ 
able  channels.  Six  are  capable  of  recording  vibration  or 
sound  up  to  5000  cps  and  six  are  capable  of  recording  high 
level  transients  from  DC  to  200  cps.  The  entire  mechanical 
spectrum  of  shock,  vibration  and  continuous  acceleration  is 
recorded  along  with  the  acoustic  levels.  Of  course,  the  re¬ 
corder  tapes  must  be  recovered  but  this  has  not  been  an 
insurmountable  problem. 

Returning  to  Figure  4,  step  3  is  in  the  nature  of 
an  exploration  for  design  errors  or  oversights.  One  or  two 
models  is  sufficient  and  they  are  run  through  all  the  en¬ 
vironments.  The  causes  of  failures  are  usually  obvious  and 
corrective  action  is  usually  not  too  difficult.  Higher  than 
expected  stress  levels  are  frequently  employed  to  afford 
some  assurance  that  the  corrective  action  has  been  adequate, 
but  admittedly  we  do  not  know  what  different  levels  mean  in 
Quantitative  terms.  The  experiment  involved  in  item  4  can 
now  be  designed. 

Before  going  into  this,  however,  it  might  be  well 
to  examine  a  frequently  proposed  alternate  procedure.  V/hy 
not  use  the  missile  environment  data  and  test  each  component, 
chassis  and  subsysten  separately  before  building  a  system  in 
order  to  insure  that  a  margin  of  from  5  to  10  standard  devi¬ 
ations  between  mean  stress  and  mean  strength  actually  exists. 
Conceivably  this  is  possible  but  usually  there  is  insuffi¬ 
cient  time  for  this  method. 

Figure  6  shows  a  guidance  unit  mounted  in  the  for¬ 
ward  section  of  its  missile.  This  particular  guidance  unit 
is  assembled  into  a  low  transmissibility  magnesium  casting 
and  all  the  electronic  packages  are  plug-in  units,  inter¬ 
changeable  between  systems.  For  this  test,  accelerometers 
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were  mounted  in  the  areas  found  previously  to  be  subject  to 
high  amplification.  The  entire  assembly  was  mounted  on  a 
shaker  and  driven  .vertically  with  a  sine  wave  to  an  amplitude 
of  2.5g  with,  control  at  the  base. 

Figure  7  shows  the  levels  observed  at  the  base 
compared  to  the  levels  observed  at  a  ring  about  8  Inches  be¬ 
low  the  guidance  unit  mounting.  Amplifications  run  over  10 
to  1  in  relatively  narrow  bands.  Figure  8  Bhows  the  various 
levels  observed  at  one  rather  critical  point,  a  gyro  mount¬ 
ing,  in  the  guidance  unit  with  g  level  control  at  the  base, 
the  ring  8  Inches  below  the  guidance  unit  and  with  the  guid¬ 
ance  unit  mounted  on  a  rigid  fixture.  We  have  made  many 
more  observations  but  these  should  suffice  to  indicate  that 
the  vibration  levels  experienced  in  a  guidance  unit  are  any¬ 
body's  guess  until  a  complete  structure  is  available  for 
test. 

Returning  to  design  evaluation,  the  purpose  is  to 
determine  if  the  reliability  inherent  in  the  design  is  in 
the  .98  to  .99  range.  Figure  9  is  a  breakdown  of  the  prob¬ 
lem  that  we  find  useful.  Under  pre-flight,  shipment  and 
handling  involve  shock  and  vibration,  non-operating.  Stor¬ 
age  involves  temperature  extremes  and  humidity,  again  non¬ 
operating.  Periodic  checkouts  involve  operating  time  for 
the  useful  life  of  the  guidance  unit  over  a  relatively  wide 
temperature  range.  We  have  found  that,  if  a  long  life  is 
to  be  expected  during  this  phase,  the  temperature  range  must 
be  restricted  to  something  reasonable,  high  temperature 
particularly  shortens  life.  All  these  situations  are  known 
and  methods  for  determining  the  probability  of  failure  in 
any  one  of  them  can  be  devised. 

Flight  reliability  involves  a  determination  of 
flight  environment  and  the  probability  of  successful  oper¬ 
ation  for  the  time  of  controlled  flight,  under  that  envlron- 
'ment.  One  might  argue  that,  as  long  as  the  flight  reliabil¬ 
ity  is  as  high  as  required,  pre-flight  reliability  is  unim¬ 
portant.  This  is  not  true  for  two  reasons.  First,  it  costs 
money  to  repair  electronic  equipment  and  second,  and  more 
Important,  there  is  no  good  way  of  assuring  that  the  relia¬ 
bility  inherent  in  the  design  is  preserved  after  numerous 
"field  repairs". 

The  determination  of  the  probability  of  failure  in 
any  of  these  situations  involves  establishing  some  pr  babll- 
lty  model  which  represents  the  physical  situation.  The 
Poisson  probability  density  function  and  the  related  expon¬ 
ential  function  have  gained  wide  usage  over  the  past  few 
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years  for  life  testing,  failure  prediction,  etc.  This  is 
shown  on  Figure  10.  It  has  two  very  important  advantages 
over  other  distributions.  First,  it  is  easy  to  use  and 
second,  it  assumes  a  constant  failure  rate.  The  second 
assumption  is  really  vital  since  it  implies  that  the 
probability  of  failure  at  any  time  is  independent  of  its 
previous  history  as  long  as  the  Poisson  process  applies 
to  this  history.  Therefore,  if  the  Poisson  process  can 
be  shown  to  represent  an  acceptable  approximation  to  ob¬ 
served  failure  data  in  each  environment  involved,  the  order 
of  subjecting  units  to  successive  environments  is  of  no 
importance.  Furthermore,  if  there  is  no  evidence  of  physical 
Interaction  between  environments,  it  can  be  assumed  that 
interaction  between  environments  applied  simultaneously  is 
unlikely.  This  Justifies  the  assumption  that  the  proba¬ 
bility  of  success  resulting  from  the  simultaneous  application 
of  several  environments  equals  the  product  of  the  probability 
for  each  applied  separately.  The  problem,  then,  is  to 
establish  that  the  Poisson  process  is  applicable  to  the 
critical  environments  involved  in  successfully  delivering  a 
missile  from  the  factory  to  its  target.  - 

Figure  11  shows  these  critical  environments  and 
where  they  are  found.  It  is  interesting  to  note  that  the 
shock  environment  associated  with  rail  transport  across  the 
country  is  frequently  far  more  severe  than  shock  associated 
with  flight.  The  vibration  environment  associated  with 
flight,  however,  is  generally  more  severe  than  that  experi¬ 
enced  in  any  common  carrier  but  it  operates  for  a  much 
shorter  time.  Transporting  a  unit  in  the  back  of  a  truck 
for  several  hundred  miles,  unprotected  by  a  suitable  ship¬ 
ping  container,  is  almost  sure  to  wreck  it.  This  means  that 
containers  with  suitable  vibration  i solation  are  usually 
necessary. 


In  the  guidance  unit  I  am  going  to  discuss,  we  are 
concerned  with  shock  during  transportation,  temperature  ex¬ 
tremes  dviring  storage,  vibration  and  altitude  during  flight 
and  ambient  during  checkout.  We  have  found  that  high  reli¬ 
ability  and  large  temperature  variations  while  operating  are 
incompatible.  Therefore,  reasonable  control  of  temperature 
during  operation,  within  or  50  degrees  F,  is  mandatory. 

The  probability  of  successful  flight  for  this  guidance  unit 
has  already  been  set  as  .985*  The  Inherent  design  capability 
must  be  at,  or  better  than,  this  level,  manufactured  product 
must  meet  this  level,  and  guidance  units  in  the  field  must 
be  capable  of  meeting  this  level  throughout  their  useful 
life.  This  last  point  is  very  important.  It  requires,  aB 
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already  stated,  that  the  need  for  field  repair  should  be 
kept  to  a  minimum  by  providing  not  only  a  high  resistance  to 
shipment,  handling  and  storage  but  also  a  mean  time  between 
failures  which  will  assure  that  only  a  small  percentage  of 
the  units  will  fail  as  a  result  of  the  normal  routine  check¬ 
out  during  a  5  year  period. 

The  prediction  studies  on  this  unit  indicated  a 
mean  time  between  failures  of  500  hours.  If  a  biweekly 
check  requiring  10  minutes  operation  is  contemplated,  it 
would  be  expected  that  /*fS  would  require  repair  during  a  5 
year  period.  This  is  reasonable,  and  a  life  test  can  be  set 
up  to  demonstrate  that  a  mean  time  between  failures  of  at 
least  500  hours  has  been  attained.  The  Poisson  function  has 
gained  general  acceptance  as  applicable  to  life  testing  of 
complex  electronic  systems  so  it  need  not  be  discussed 
farther. 

In  the  flight  environment,  it  has  been  our  experi¬ 
ence  that  vibration  is  by  far  the  most  critical  factor. 

This  cannot  be  accepted  as  applicable  to  any  electronic 
design  in  any  missile  structure  but  it  has  proven  true  in 
those  we  have  been  concerned  with.  It  is  also  logical  since 
the  response  of  an  assembly  to  a  shock  impulse  is  a  highly 
damped  wave  starting  at  a  high  amplitude  over  a  broad  fre¬ 
quency  band.  It  only  excites  resonances  for  a  fraction  of  a 
second  compared  to  a  relatively  continuous  excitation  due  to 
vibration.  The  inference  that  immediately  results  is  that 
the  excitation  during  the  vibration  test  should  be  preferably 
an  actual  tape  produced  from  a  combination  of  several 
flights  or,  as  a  compromise,  band  limited  noise  that  approx¬ 
imates  such  a  recording.  With  this  decision  made,  it  re¬ 
mains  to  be  demonstrated  that  the  Poisson  process  represents 
an  acceptable  approximation  to  the  failure  time  distribu¬ 
tions  observed. 

Several  methods  are  available,  the  Kolmogrov- 
Smirnov  test  which  is  quite  satisfactory  with  20  or  30 
failures  or  the  Bartholomew  test  which  appears  to  be  very 
powerful  with  200  or  more  failures.  However,  these  tests  do 
not  give  a  picture  and  engineers  usually  like  pictures. 

Figure  12  indicates  the  method  we  have  found  most  convincing 
to  engineers,  although  statistically  it  leaves  something  to 
be  desired.  The  curves  are  boundaries  of  a  90°/o  confidence 
interval  plotted  on  either  side  of  the  L5  degree  line  which 
represents  the  mean  time  between  failures  of  the  unit  under 
test.  The  unit  is  run  on  a  shaker,  in  this  instance  at 
2-l/2g  Level  with  a  swept  sine  until  20  to  30  failures  are 
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observed,  the  time  of  each  failure  being  recorded.  The 
times  between  failures  are  then  ordered  and  the  number  of 
failures  occurring  at  any  time  are  plotted  cumulatively 
against  the  expected  number  which  is  calculated  from  the 
Poisson  Function.  If  all  the  points  fall  within  the  limit 
lines,  the  process  can  be  considered  sufficiently  close  to 
the  Poisson  for  the  latter  to  be  used  for  estimating  pur¬ 
poses.  In  this  particular  instance,  2  units  were  involved 
and  some  30  hours  of  shaker  time.  In  other  instances,  we 
have  run  considerably  longer  and  the  Poisson  Model  has  al¬ 
ways  been  applicable. 

With  this  established,  it  is  only  necessary  to  run 
a  test  long  enough  to  demonstrate  that  the  mean  time  between 
failures  required  to  give  the  desired  probability  of  success 
in  flight  has  been  attained.  The  other  environments  in¬ 
volved  in  flight  are  altitude  and  continuous  acceleration. 
Actually,  we  have  never  been  able  to  run  either  of  these 
tests  long  enough  to  demonstrate  that  a  Poisson  failure  pro¬ 
cess  is  operating.  We  just  do  not  get  failures  in  a  good 
design.  However,  we  assume  such  a  process. 

The  derivation  of  the  criteria  associated  with  the 
flight  environments  i3  shown  in  Figure  13.  The  required 
probability  of  success  is  .935.  An  equal  probability  of 
failure  for  each  of  the  three  environments  of  .01  was 
assumed  and  the  criteria  based  upon  that.  In  order  to  make 
such  a  criteria  acceptable  ir  production,  it  would  be  nec¬ 
essary  to  have  a  design  capability  much  better  than  this. 

A  .90  probability  of  acceptance  of  any  one  sample  was 
assumed  which  means  that  the  actual  level  has  to  be  at  least 
as  good  as  .001*3  shown  in  the  third  column.  The  total 
flight  reliability  then  is  .98?  which  meets  the  objective. 

Figure  1U  shows  the  criteria  as  they  apply  to  a 
sample  of  f>  systems.  For  vibration,  the  test  is  run  with 
random  noise  at  g-level  of  .01g2  per  cycle.  The  test  is  run 
for  20  minutes  in  e;  ch  of  3  mutually  perpendicular  planes. 

If  no  failures  are  observed,  the  mean  time  between  failure 
for  a  single  system  is  met  at  a  90>  confidence  level  for  a 
single  unit  and  the  test  is  over.  A  table  is  provided  which 
gives  the  acceptance  and  rejection  points  associated  with 
the  9(X  level.  If  no  decision  can  be  made  up  to  30  fail¬ 
ures,  the  mean  time  obtained  by  dividing  the  total  time  by 
30  is  used  for  determining  conformance. 

The  criteria  for  altitude  and  continuous  acceler¬ 
ation  are  also  based  upon  the  time  of  exposure  but  only  a 
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single  rejection  criterion  is  used.  Failures  are  not  ex¬ 
pected  and  seldom  observed  in  this  area. 

The  pre-flight  reliability  criteria  are  shown  on 
Figure  15.  These  criteria  are  combined  in  a  manner  similar 
to  the  flight  reliability  criteria  and  result  in  a  pre- 
flight  reliability  of  .95  for  a  5-year,  period.  The  criterion 
for  mean  time  on  the  life  test  is  operated  in  the  same  manner 
as  that  for  vibration,  accept  on  90%  confidence  of  meeting 
the  individual  level,  reject  on  90%  confidence  of  failing, 
or  run  until  30  failures  are  observed. 

The  shock  test  involves  10  drops  at  a  response 
level  of  at  least  lOOg  between  100  and  700  cps  in  each  of  3 
planes,  a  total  of  30  drops  for  each  system.  With  150  drops 
on  5  systems  and  allowing  3  failures  in  the  total  sample,  we 
feel  reasonably  sure  of  meeting  a  probability  of  success  of 
•98  or  better.  Actually,  this  drop  test  severity  is  likely 
to  be  equalled  only  1 %  of  the  time  in  3000  miles  of  rail 
transport.  The  temperature  test  uses  the  usual  Mil- 
specification  range  of  -80°F  to  l60°F  but  it  is  run  for  10 
cycles.  Failures  are  not  expected  in  this  test  but  we  have 
assumed  a  .01  probability  of  failure  to  be  adequate.  The 
rate  of  temperature  change  and  time  of  dwell  at  the  extremes 
are  set  to  achieve  this  probability  in  actual  service,  stor¬ 
age  or  plane  transportation .  A  sample  of  5  meeting  both  the 
pre-flight  and  flight  criteria  supplies  adequate  assurance 
that  the  design  is  capable  of  meeting  the  .985  reliability 
objective  required. 

With  the  design  capability  established,  the  pro¬ 
duction  control  test  can  be  set  up  by  using  the  criteria  for 
an  individual  system.  It  Is  not  practicable  In  the  usual 
small  sampling  of  a  production  process  to  require  criteria 
to  apply  to  a  group  of  several  systems.  The  minimum  sample 
size  and  distribution  of  the  sample  is  given  on  Figure  16. 
This  is  an  empirical  rule  and  the  best  that  can  be  said  for 
it  is  that  it  has  been  proven  by  experience.  This  formula 
Is  applicable  to  cor. tracts  requiring  production  intervals  of 
one  year  or  less.  For  intervals  of  greater  duration,  this 
can  usually  be  considered  to  apply  on  a  per  year  basis. 
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THE  SAMPLE  SHALL  BE  DISTRIBUTED  APPROX¬ 
IMATELY  UNIFORMLY  OVER  THE  PRODUCTION  PERIOD. 

,  „  Figure  16 


RELIABILITY  CONCEPTS  APPLIED  TO  OBTAIN 
DESIGN  SPECIFICATIONS  FROM  MEASURED 
ENVIRONMENTAL  BATA 

Elliot  H.  Kahn,  The  W.L.  Maxson  Corporation 
Samuel  Stempler,  The  W.L.  Maxson  Corporation 

ABSTRACT 

The  Reliability  Analysts  Section  of  The  W.L.  Maxson  Corporation 
has  prepared  a  study  on  how  raw  measured  environmental  data  was 
used  in  the  preparation  of  a  design  specification.  Such  techniques  are 
believed  applicable  and,  indeed,  desirable  when  starting  a  research 
and  development  project. 

This  article  indicates  how  the  vibration  requirements  for  the 
structure  of  a  unit  were  determined  in  a  specific  case:  an  aluminum 
alloy  casting  used  as  the  primary  structure  in  a  critical  guided  mis¬ 
sile  safety  and  arming  device.  The  Scope  of  Work  listed  limited 
measured  data,  giving  steady- state  loads,  vibration,  and  shock 
loads.  A  review  of  this  data  indicated  that  the  steady- state  and  shock 
loads  were  not  critical  compared  with  the  vibration  loads. 

For  the  case  in  question,  the  vibration  design  levels  are  deemed 
to  be  influenced  by  four  considerations: 

1.  Safety  factors. 

2.  Probability  of  exceeding  measured  data. 

3.  Resonances  in  the  structure. 

4.  Factors  of  ignorance. 

These  factors  are  discussed  in  greater  detail,  and  a  final  recom¬ 
mendation  for  a  vibration  design  specification  is  made. 

RELIABILITY  CONCEPTS  APPLIED  TO  OBTAIN  DESIGN  SPEC¬ 
IFICATIONS  FROM  MEASURED  ENVIRONMENTAL  DATA 

What  reliability  considerations  are  involved  in  using  measured 
environmental  data  for  the  preparation  of  a  design  specification?  The 
Reliability  Analysis  Section  of  The  W.L.  Maxson  Corporation  recent¬ 
ly  was  required  to  answer  this  question  for  a  project  charged  with  the 
design  of  a  device  tor  a  guided  missile.  The  correction  factors  be¬ 
lieved  necessary  before  limited  environmental  data  can  be  used  reli¬ 
ably  are  presented  herein.  The  techniques  employed  are  believed 
applicable  and,  indeed,  desirable  when  starting  a  research  and  devel¬ 
opment  project. 
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Environmental  requirements  obviously  must  be  included  in  contracts 
for  military  equipment.  Equipment  must  operate  or  be  capable  of  being 
stored  under  these  environmental  stresses.  Often,  the  environments 
stipulated  in  the  specifications  referenced  in  the  contract  have  been  so 
generalized  that  no  exact  relationship  can  be  found  between  them  and 
the  actual  environment  to  which  the  equipment  will  be  subjected.  The 
limits  specified  are  an  attempt  to  insure  that  the  equipment  meets  cer¬ 
tain  minimum  standards  of  ruggedness,  durability,  and  useful  life. 
However,  measured  environmental  data,  when  available,  can  and  should 
augment  the  specification. 

Paradoxically,  the  availability  of  actual  environmental  data  may 
complicate  the  role  of  the  reliability  engineer-in  relation  to  the  design 
engineer.  This  is  so  because  the  measured  environmental  data  is  often, 
of  necessity,  obtained  from  a  statistically  insignificant  sample  under 
conditions  that  aggravate  the  measurement  problems,  and  should  not 
be  used  directly.  However,  the  data  obtained  from  measured  environ¬ 
ments  is  necessary,  and  can  be  reasonably  adjusted  to  enhance  the 
probability  of  obtaining  the  desired  reliability  goal.  This  article  indi¬ 
cates  the  manner  in  which  the  vibration  requirements  for  the  structure 
of  a  unit  was  determined  in  a  specific  case.  Figure  1  illustrates  how 
measured  data  could  be  used  as  a  base  upon  which  various  factors  can 
pyramid  to  "build”  a  good  design  specification. 

We  have  taken  the  liberty  of  making  a  number  of  numerical  simpli¬ 
fications  without  specifically  noting  them,  because  a  rigorous  mathe¬ 
matical  treatment  of  the  subject  is  not  germane  to  the  topic. 

The  background  of  the  problem  is  as  follows:  an  aluminum  alloy 
casting  is  used  as  the  primary  structure  in  a  guided  missile  adaption 
kit.  The  Scope  of  Work  contained  limited  measured  environmental 
data  that  stipulated  steady -state  loads,  vibration  loads,  and  shock 
loads.  A  review  of  this  data  indicated  that  the  steady-state  and  shock 
loads  were  not  critical,  compared  with  the  vibration  loads. 

In  detail,  the  nature  of  the  vibration  specification  was  that  the  unit 
had  to  be  capable  of  functioning  with  applied  vibration  levels  up  to  a 
peak  of  17g,  which  was  the  approximate  flight  and  handling  environ¬ 
ment.  Analysis  of  limited  flight,  data  indicated  that  the  vibration  had 
an  rms  value  of  lOg,  with  peaks  of  17g  actually  observed.  A  high 
operational  reliability  (99.  5  percent)  was  specified  for  the  unit,  with  a 
much  higher  probability  of  functioning  as  a  safety  mechanism  specified. 

In  evaluating  techniques  to  adapt  this  data  into  a  reliability  design, 
it  became  apparent  that  Lusser,  in  "Reliability  Through  Safety  Mar¬ 
gins”  (October  1958,  Research  and  Development  Division,  Army  Rocket 
and  Guided  Missile  Division,  Redstone  Arsenal),  had  outlined  most  of 
the  factors  that  lead  to  a  selection  of  safety  margins  in  a  reliability 
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FACTORS  OF  IGNORANCE 

(Location of  resonances,  stress 
concentrations) 

''GAMBLE"  FACTOR 

(%  of  time  designer  Is  willing  to  exceed 
design  level) 

MATERIAL  VARIATIONS 

(Including  fabrication  procedures  and 
tolerance  buildup) 

SELECTION  OF  PROPER  POINTS  TO  MONITOR 

(Limited  number  of  telemetry  channels  and  hand- 
width  available) 

SIMILARITY  BETWEEN  TELEMETER  DEVICE  AND  FINAL  DEVICE 


(Modifications  to  carry  telemetry  hardware  and/or  design  modlflca 
tions) 


•nr* 


specification.  Although  these  principles  may  be  rather  provocative 
to  the  electrical  engineer ,  they  have  been  applied  for  many  years  in 
such  civil  engineering  fields  as  traffic  flow,  flood  control,  and  struc¬ 
tural  safety  factors.  Typical  papers  in  the  field  include  Freudenthal's 
"The  Safety  of  Structures” ,  Vol.  112,  1947  Transactions  of  the 
American  Society  of  Civil  Engineers,  which  reasserts  the  concept  of 
safety  factors;  and  various  issues  of  highway  and  bridge  specifications, 
which  accept  probability  concepts  in  computing  maximum  design  loads 
and  arrangement  of  vehicles  on  a  bridge.  Naturally,  a  measure  of 
risk  is  involved,  and  we  are  all  aware  of  structures  that  collapse  when 
subjected  to  conditions  of  severe  overload.  Generally,  the  failure  rate 
allowed  in  these  structures  is  based  on  economic  principles.  In  mili¬ 
tary  applications,  operations  research  adds  other  considerations;  and 
Ihe  designation  of  allowable  failure  rates  Is  based  upon  the  more  in¬ 
volved  factors  that  influence  the  nation's  security.  In  some  of  our 
critical  nuclear  programs,  these  failure  rates  are  so  small  that  they 
actually  may  be  Interpreted  as  stating  that  no  failure  rate  is  tolerated 
(that  is,  absolute  safety  is  required). 

For  the  case  in  question,  the  vibration  design  levels  were  deemed 
to  be  influenced  by  four  considerations. 

1.  Safety  factors  (which  relate  to  the  variations  possible  in  the 
materials,  fabrication  processes,  inspection  procedures,  and 
usage  conditions). 

2.  Probability  of  exceeding  measured  data  (purely  on  a  statistical 
basis) . 

3.  Resonances  in  the  structure  (and  other  factors  associated  with 
the  absence  oi  a  laboratory-controlled  experiment,  which 
actually  relates  measured  data  to  what  is  happening  in  the  unit). 

4.  Factors  of  Ignorance  (which,  paraphrasing  Freudenthal,  relates 
to  the  limitations  in  our  intellectual  concepts  in  understanding 
and  describing  what  is  happening). 


Applications  of  the  above-mentioned  considerations  affected  the 
design  specifications  as  f  illows: 

1.  Safety  Factors. 

It  has  been  reported  (reference  that  the  fatigue  strength  for 
aluminum  alloy  (castings)  has  a  scatter  band  ♦  20  percent  about  the 
average  value.  Generally,  statisticians  consider  the  "limits  of  con¬ 
sideration"  to  be  at  the  three-sigma  point  (reference  B). 
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Let  a  =  sigma  =  standard  deviation 
Sav  =  average  strength 
Sdl  -strength  required  at  design  level 
Then,  20  percent  of  Sav  =  the  three -sigma  point,  or  <7=  0.2SaY 


A  reliability  code  is  being  prepared  by  one  of  the  Arsenals  prefer¬ 
ence  C)  which  contains  the  requirement  that  the  contractor  shall  prove 
that  a  safety  margin  of  at  least  five  sigma  is  available. 

Therefore,  the  desien  point  should  be  So  away  from  the  average 
strength,  or 


SjJl  -  Say  •  5  (J  =  Sav  ”  1/3  Sft v 

Sdl  «  2/3  sav 

The  ratio  of  Sav  -1.5 

Sdi 

The  average  strength  should  be  1.  5  times  the  strength  required. 

On  this  basis,  the  vibration  design  level  should  be  increased  from 
the  specified  17g  by  a  1. 5  factor  to  25.  5g.  .. 

For  similar  computations,  a  family  of  curves  is  available  'refer¬ 
ence  D)  which  relates  the  safety  factor,  for  parlous  ratios  of  Sav  to 
Sdl,  to  the  sigma  safety  margin. 

When  one  discusses  safety  factors  for  machines,  aircraft,  and 
structures,  typically,  a  safety  factor  of  only  1.  5  is  specified  (refer¬ 
ence  C) .  Although  it  has  been  developed  from  specifications  for 
structures,  this  low  safety  factor  merely  allows  for  the  known  vari¬ 
ations  of  the  basic  materials ,  not  for  the  many  additional  uncertainties 
and  contingencies  that  always  affect  the  components  of  even  seemingly 
simple  military  equipment. 

2.  Probability  of  Exceeding  Measured  Data. 

Assuming  that  the  vibrations  are  random  and  the  vibration  levels 
have  a  normal  distribu  ion,  the  probability  of  exceeding  the  rms  value 
of  the  vibrations  can  be  determined.  If  one  stipulates  that  the  design 
level  of  vibration  should  not  be  exceeded  more  than  one  percent  of  the 
time,  then  the  design  level  should  be  approximately  2. 6  times  the 
expected  rms  level.  Therefore,  the  design  for  the  problem  posed 
above  should  required  (10)  x  (2. 6),  or  26g,  if  we  are  not  to  exceed  our 
design  level  of  vibration  more  than  one  percent  of  the  time.  With  the 
aid  of  tables  available  In  most  books  on  statistics,  the  design  levels 
can  be  specified  on  the  basis  of  their  not  being  exceeded  for  whatever 
percentage  of  lime  the  designer  believes  to  be  significant.  Actually, 
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fatigue  damage  is  cumulative,  and  a  detailed  application  of  Miner's 
approach  could  be  followed.  This  would  integrate  the  damage  caused 
by  the  different  vibration  levels  for  the  portion  of  the  mission  time 
present.  A  brief  summary  of  the  approach  is  given  in  reference  E. 

3.  Resonance. 

It  should  be  fully  expected  that  several  points  in  the  unit  will 
experience  amplification  of  applied  vibration  levels  due  to  resonances. 
However,  it  is  almost  impossible  to  calculate  resonance  effects  for 
a  structure  as  complicated  as  a  typical  missile  casting. 

The  nature  of  the  design  led  to  the  presumption  that  an  amplifl- 
cation  factor  of  at  least  1.  5  was  almost  certain  to  be  present  at  points 
within  the  unit.  Consequently,  requiring  components  and  the  structure 
to  be  designed  for  such  amplification  is  entirely  reasonable,  and  does 
not  represent  an  ultra-conservative  or  over -designed  approach.  This 
magnification  factor  is  not  based  on  any  theoretical  work  but  is  merely 
what  we  have  found  on  similar  units  in  the  past. 


4.  Factors  of  Ignorance. 

There  are  several  additional  considerations  to  the  problem. 
Broadly,  they  may  be  lumped  into  what  has  been  termed  factors  of  igno¬ 
rance.  For  example,  if  the  most  highly  stressed  points  are  not  affected 
by  resonance,  the  previously  computed  factor  of  1.  5  (which  raised  the 
vibration  level  to  25.  5g  to  allow  for  material  variations)  is  valid.  How¬ 
ever,  if  this  stress  is  subjected  to  a  1.  5  magnification  because  of  reso¬ 
nance,  the  correct  design  vibration  level  is  38. 3g.  Naturally,  it  is 
perfectly  valid  to  question  the  probability  of  obtaining  a  location  with 
a  high -resonance  condition  combined  with  low  material  strength. 

Similar  questions  arise  as  to  the  probability  of  detecting  major 
flight  loads  in  a  limited  flight  program  with  limited  pickups,  and  the 
ability  to  determine  the  relationship  between  the  data  obtained  from  a 
telemetry  missile  and  an  actual  missile  having  the  unit  installed.  Obvi¬ 
ously,  if  all  these  conditions  were  included  (presuming  they  could  be 
evaluated,  even  by  an  rms  technique)  the  final  design  levels  would  be 
much  higher  than  the  levels  specified. 

It  must  be  recognized  that  it  is  impossible  (and  not  valid)  to 
compound  probability  on  probability  without  some  overall  evaluation  of 
the  statistical  aspects.  This  evaluation  should  cover  specification  re¬ 
quirements,  flight  data,  materials,  experience  (on  the  basis  of  design 
standards),  possible  reduction_in  reliability  elsewhere  (because  of  ex¬ 
cessive  weight  and  volume  allocated  to  the  structure  at  the  expense  of 


other  components),  and  so  forth.  Accordingly,  based  on  such  con¬ 
siderations  as  material  variations,  probability  of  exceeding  a  specified 
"g"  level,  and  resonances  observed  in  similar  devices  (past  experience), 
a  factor  of  1.  5  times  the  maximum  observed  flight  load  of  17g  was 
selected  to  obtain  design  levels  for  the  device.  It  was  felt  that  the  de¬ 
sign  would  be  jeopardized,  in  the  light  of  current  experience,  if  this 
safety  factor  of  at  least  1.  5  were  not  used.  On  the  other  hand,  it  was 
also  felt  that  the  designer  should  be  aware  that  a  safety  factor  as  low 
as  1. 5  could  be  questioned  for  adequacy,  since  nothing  had  been  allowed 
for  factors  of  ignorance. 

In  performing  this  analysis,  it  was  recognized  that  only  elementary 
numerical  computations  were  used,  rather  than  more  sophisticated 
approaches  to  the  probability  distribution.  Although  not  necessarily 
valid,  the  stated  reason  is  that  the  factors  of  ignorance  are  so  great 
(pui  ticularly  at  the  start  of  a  research  and  development  program) that 
it  was  not  considered  worthwhile  to  go  into  the  subject  more  deeply. 

Actually,  the  application  of  reliability  factors  to  the  design  levels 
was  questioned  on  the  basis  that  other  sections  of  the  missile  were  not 
being  designed  to  increased  levels.  In  a  way,  this  did  make  the  adaption 
kit  "stronger”  than  the  missile.  However,  the  adaption  kit  is  expected 
to  perform  its  function  even  while  the  missile  may  be  structurally  fail¬ 
ing  in  flight.  And  the  device  must  perform  with  a  reliability  greater 
than  that  expected  of  the  rest  of  the  missile. 

Consequently,  in  discharging  its  responsibility  of  determining  de¬ 
sign  and  test  specification  criteria  for  this  device,  the  Reliability 
Analysis  Section  of  Maxson  combined  the  measured  environmental  data 
and  the  required  reliability  into  a  compatible  specification  for  design 
purposes. 

Three  levels  of  environmental  severity  were  specified.  These 
levels,  Type  A,  Type  B,  and  Type  C,  were  defined  as  follows  (quoting 
directly  from  the  prepared  design  specification): 

Type  A:  "Type  A  Environment  (Contract  Minimum  Value):  These 
levels  are  directly  abstracted  from  the  contract.  Every 
item  must  be  capable  of  withstanding  these  levels" . 

Type  B:  "Type  B  Environment  (Design  Target  Value):  These  levels 
represent  environments  which  may  exceed  those  specified 
In  the  contract.  All  components  and  subassemblies  must 
be  capable  of  withstanding  these  environments.  The 
completed  unit  must  be  exposed  to  these  environmental 
levels,  ancl  the  effects  evaluated,  before  a  design  can  be 
considered  to  meet  the  reliability  provisions  of  the  con¬ 
tract.  Items  subjected  to  Type  B  environment  are  not 
suitable  for  stockpiling  or  delivery  to  the  customer." 
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Type  C:  "Type  C  Environment  (Destruction  Value):  The  margin  of 
safety  inherent  In  the  design  must  be  established  by  ex¬ 
posure  to  these  destructive  environments.  These  environ¬ 
ments  exceed  the  Type  B  environments.  The  levels  stated 
In  this  specification  are  based  upon  the  capabilities  of 
presently  available  test  equipment.  Should  Increased 
capability  become  available,  then  advantage  should  be  taken  • 

of  the  opportunity  to  Induce  greater  environmental  stress. 

The  Type  C  environment  Is  applicable  to  components,  sub¬ 
assemblies  and  the  completed  unit.  Items  subjected  to  the 
Type  C  environments  are  not  suitable  for  stockpiling  or 
delivery  to  the  customer." 

It  is  Intended  that  each  of  the  different  levels  of  environmental  stresses 
w<”  help  achieve  the  objectives  outlined  below. 

The  Type  A  environment  Is  the  formal  test  environment,  which  must 
be  met  before  the.  customer  accepts  any  equipment. 

The  Type  B  environment  permits  all  subassemblies  to  be  tested 
and  evaluated  In  parallel  programs  without  waiting  for  final  assembly. 

It  permits  a  minimum  amount  of  intensification  of  the  contractually 
specified  environments  (which  apply  to  the  system)  to  allow  for  such 
quantities  as  vibration  magnification,  equipment  heating,  and  other  re¬ 
lated  items.  It  Is  recognized  that  the  Individual  components  and  sub- 
assemblies  must  have  a  greater  reliability  than  that  required  of  the  sys¬ 
tem  as  a  whole.  Consequently,  component  specialists,  who  must  select 
specific  components  and  vendors,  can  use  the  Type  B  environments  as 
a  basis  for  negotiating  with  outside  sources. 

The  Type  C  environment  is  Bell- explanatory  and  is  in  the  spirit  of 
Lusser's  recommendation  to  determine  safety  marginnot  all  equipment. 

Should,  for  one  reason  or  another,  the  operating  requirements  for  the 
system  change,  knowledge  of  the  safety  margins  would  be  Invaluable. 

Evaluation  and  determination  of  changes  (If  any)  could  be  rapidly  made. 

In  our  approach  to  this  problem,  we  have  felt  that  reliability  Is 
applicable  to  all  aspects  of  the  design,  both  electrical  and  mechanical. 

Indeed,  an  engineering  solution  that  excludes  reliability  considerations 
anywhere  Is  Incomplete  and  should  be  treated  as  such.  ' 

APPENDIX 

Subsequent  events  tend  to  justify  the  approach  described  In  this 
article.  A  missile,  containing  the  first  deliverable  adaption  kit  (which 
was  designed  using  the  approach  described)  broke  apart  shortly  aiter 
launeh.  Telemetered  Information  indicates  that  the  safety  and  arming 
functions  ol  the  device  were  fulfilled  as  designed.  Who  knows  what 
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the  environment  wae  at  the  time?  Yet  this  was  p-ec 
the  equipment  had  to  perform. 
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QUALITY  CONTROL  AND  MISSILE  SYSTEMS  ACCEPTANCE  TESTING 


NAME i  D.  K.  Meikle 

GO’fP^rv,  Chrysler  Corporation  Missile  Division 

Missile  Systems  Acceptance  Testing  as  establi-shed  in  the  Redstone 
and  Jupiter  Missile  Programs,  refers  to  the  Electro-Mechanical  Testa 
performed  on  the  Missiles  and  Ground  Support  Equipment  as  a  part  of 
+he  Quality  Control  function.  These  tests  are  run  at  the  component 
level  prior  to  installation,  and  at  the  component  ,  sub-system  and  oom- 
plete  systems  level  alter  installation.  Quality  Control  Engineers 
write  the  detailed  test  procedures  used  for  theso  tests  and  Quality  Control 
personnel  perform  them.  The  testa  accomplish  three  main  objectives;  they 
assure  quality  of  manufacture,  functionability  of  the  system,  and  com¬ 
patibility  between  the  missile  and  ita  firing  equipment. 

Assuring  quality  in  the  product  ia  accomplished  by  testing  each 
valve,  relay,  computer,  actuator,  distributor,  gyro,  etc.,  before  assembly 
Into  a  panel  or  black  box  to  see  that  it  performs  to  design  specifica¬ 
tions.  Then  it  is  tested  again  after  assembly  into  the  panel  or  black 
box,  and  again  in  final  checkout  after  installation  in  the  missile  or  in 
a  vehicle.  These  later  tests  insure  that  no  damage  has  resulted  during 
assembly  and  that  ita  new  environment  does  not  deter  the  components’ 
ability  to  meet  the  original  design  specifications. 

It  should  be  established  that  Chrysler  MiBSile  does  not  do  100% 
inspection  or  testing.  A  better  figure  to  UBe  would  be  400%  as  a  minimum 
on  most  functional  components,  and  600%  or  80C%  on  many  others.  Statis¬ 
tical  sampling  is  used  only  at  receiving  inspection  on  large  lots  of  raw 
material  or  large  quantities  of  small  hardware  and  components  (screws, 
rivets,  resistors,  etc.)  and  in-process  inspection  of  some  fabricated 
non-functional  parts. 

Assuring  functionability  in  the  missile  is  accomplished  by  verify¬ 
ing  that  the  missile  will  perform  properly  during  flight.  By  establi¬ 
shing  teat  methods  which  duplicate  or  simulate  conditions  encountered 
in  actual  flight,  the  tests  are  a  double  check  on  the  missile  system 
design.  This  is  especially  important  during  the  R  &  D  phase  of  the 
project  where  the  design  is  constantly  changing  in  an  effort  to  improve 
leliabiiity  and  performance.  But  the  early  tactical  version  also  under¬ 
goes  modifications  which  result  from  the  conditions  encountered  in  field 
me,  and  added  Bafety  and  reliability  features.  Each  change  in  the  missile 
d  isign  must  be  checked  thoroughly  to  insure  that  it  will  accomplish  the 
dt  sired  result  and  to  verily  that  each  modification  does  not  cause  inter- 
f<  rence  with  any  of  the  other  systems  aboard  the  missile. 

In  the  final  checkout  testing,  the  missile  goes  through  a  launch 
co mtdown  and  simulated  flight  as  the  final  test,  operating  every com¬ 
ponent  except  firing  up  the  engine.  The  stabilized  platform  is  set.  in  a 
gin  balled  turn  tilt  stand  which  can  rotate  it  in  any  direction  to  simulate 
mi®ile  flight  and  check  the  response  of  the  control  system  as  error  signals 
are  introduced,  ^  _  £68 


Assurance  of  compatability  with  the  actual  launching  equipment  is 
accomplished  by  designing  the  final  checkout  equipment  functionally 
identical  to  the  portable  launching  equipment  which  checks  out  and  fires 
the  missile  in  the  field.  Many  of  the  tests  performed  in  checkout  are 
also  identical  with  those  performed  in  the  field  previous  to  firing.  Many 
panels  used  in  the  checkout  station  can  be  interchanged  with  panels  in 
the  field  equipment.  As  the  missile  design  is  modified,  the  test  equip¬ 
ment,  procedures  and  specifications  used  in  checkout  must  be  changed 
+o  accomodate  the  design.  These  changes  are  at  the  same  time  incorpo¬ 
rated  into  the  missiles,  ground  support  equipment,  and  procedures  used 
-in  the  field.  In  this  manner  the  design  of  the  launching  equipment 
remains  up  to  date  and  is  compatible  with  the  missile  as  modifications 
arc  incorporated. 

As  a  further  check  on  compatibility  the  actual  ground  support  equip¬ 
ment  for  each  launch  site  is  assembled  and  mating  testB  with  each  missile 
are  performed  at  the  contractor  site.  These  tests,  under  field  environment, 
further  insure  the  compatibility  of  missile  and  ground  equipment, 

Functtonabillty,  quality,  and  compatibility  are  the  goals  established 
for  acceptance  testing,  but  why?  It  is  a  simple  matter  of  reliability. 

For  simplicity,  break  total  reliability  down  into  three  main  factors, 
design  reliability,  manufacturing  reliability,  and  field  reliability. 

Design  reliability  is  the  establishment  of  component  and  system 
parameters  and  specifications  which  when  integrated  will  define  a  wea¬ 
pons  system  capable  of  delivering  a  given  payload  to  within  a  specified 
target  area.  It  is  the  configuration  established  on  paper  and  proven 
in  lab  environmental  testing  and  initial  R  &  D  firings. 

Manufacturing  reliability  is  the  assurance  that  the  product  con¬ 
forms  to  the  specifications  established  by  the  design.  This  is  Quality 
Control,  the  evaluation  of  the  hardware  to  see  that  it  is  built  to  the 
design  specifications. 

But  complete  10055  reliability  of  design  andof  manufacture  is  not 
enough.  Field  reliability,  competant,  trained  personnel  and  procedures 
are  required  to  complete  the  reliability  of  the  system.  Proper  hookup 
of  equipment,  accruate  laying  or  aiming,  correct  preset  values,  exact 
sequence  of  operations  all  are  required. 

Thus  it  is  the  product  of  the  three  factors,  design  reliability, 
manufacturing  reliability,  and  field  reliability ’which  determines  the 
overall  reliability  of  the  missile  weapons  system.  In  establishing 
Acceptance  Testing  criteria  which  determines  the  functionability  of 
the  system,  a  partial  verification  of  design  reliability  results.  Re¬ 
peated  tests  to  insure  quality,  establishes  manufacturing  reliability, 

A  constant  check  for  compatibility  of  equipment  and  procedures  plays  a 
major  role  in  securing  field  reliability.  Therefore,  the  continued 
effort  to  maintain  high  functionability,  quality,  and  compatibility 
standards  in  acceptance  testing  has  played  n  major  role  in  establishing 
the  Redstone  and  Jupiter  as  two  of  the  rooct  reliable  of  this  nation’s 
Weapons  Systems, 
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Before  going  into  detail  on  the  exact  nature  of  the  testing  and 
test  equipment,  it  should  be  established  how  this  function  fits  into  the 
Quality  Control  organization.  The  Branch  Manager  of  Quality  Control 
reports  directly  to  the  Missile  Divfe  ion  General. .Manager,  He  is  on  the 
aa"^  level  as  the  Manufacturing  Manager,  Production  Control  Manager  and 
Chief  Engineer,  Reporting  to  the  Branch  Manager  of  Quality  Control  are 
five  line  sections.  These  consist  of  a  fatality  "Control  Engineering 
Section  which  is  responsible  for  the  planning  of  inspection  operations, 
three  inspection  sections  covering  Receiving  Acceptance,  In-Process 
Inspection,  and  Final  Acceptance  Testing;  and  a  Quality  Assurance  Section 
which  is  responsible  for  the  measuring  of  quality  achievement.  There  is 
also  a  Material  Review  Department  which  deals  th  the  disposition  of 
material  which  deviates  from  specifications. 

Quality  Control  Engineering  is  a  phase  of  engineering  that  is 
associated  strictly  with  Quality  Control,  These  are  the  people  who  plan 
:.M  the  activities  that  are  conducted  by  the  Quality  Control  Branch, 

During  the  design  and  prototype  phases  of  the  missile  program,  Quality 
Control  engineers  work  closely  with  product  design  engin  eering  groups 
in  order  to  develop  inspection  and  test  methods,  and  design  and  procure 
necessary  test  equipment.  Each  component  of  the  missile  system  is 
evaluated  to  determine  the  most  economical  and  reliable  method  for  con¬ 
trolling  its  quality.  Inspection  points  are  established  by  Quality  Control 
concurrently  with  the  preparation  of  manufacturing  routing  sheets  by  Master 
Mechanics. 

Quality  Control  Engineers  investigate  potential  suppliers  prior  to  the 
placing  of  purchase  orders  in  Older  to  assure  their  ability  to  produce 
quality  components.  Suppliers*  quality  control  facilities,  techniques, 
and  past  quality  performance  records  are  thoroughly  evaluated  and  pro¬ 
visions  are  made  for  test  samples,  material  certifications,  and  source 
inspection  as  necessary. 

In  addition  to  this,  a  very  important  function  of  Quality  Control 
Engineering  is  to  correct  quality  problems  to  see  that  they  are  not 
repeated.  This  involves  the  investigation  of  problems  reported  by 
Quality  Assurance  Section  as  a  result  of  the  occurence  of  repetitive 
defects.  They  will  determine  the  exact  cause  of  the  problem,  and  take 
the  necessary  steps  to  correct  the  problem  by  initiating  a  change  in 
inspection  method  or  recommending  a  change  in  production  process  or 
change  in  the  design  specification. 

The  most,  important  function  of  Quality  Control  is  that  of  inspection. 
The  Receiving  Acceptance  Inspection  Section  is  responsible  for  the  inspec¬ 
tion  and  test  of  all  parts  produced  by  any  of  our  suppliers.  In  some 
cacea  our  ;>eoplo  are  stationed  at  the  suppliers  *  own  plant  In  order  to 
assure  that  parts  produced  by  the  supplier  meet  the  highest  quality 
requirements.  In  general,  primary  reliance  is  placed  on  the  supplier's 
own  quality  control  system  for  usn  irinr.  the  maintenance  of  high  quality. 

At  the  present,  time,  there  are  permanent,  supplier  liaison  repreoontativeo 
stationed  at  20  of  the  moot  important  subcontractors  and  vendors.  These 
representatives  verify  the  performance  of  tho  supplier's  quality  control 
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trends,  pin-pointing  quality  problem  areas,  and  reporting  trouble 
spots  so  that  appropriate  corrective  action  can  b e  taken  either  by 
design,  production,  or  inspection. 

In  order  to  continually  evaluate  the  activities  of  our  own  quality 
control  system,  the  Quality  Assurance  Section  also  is  responsible  for 
performing  quality  audits.  These  consist  of  spot  checks  of  the  effi¬ 
ciency  of  our  inspection  and  test  operations  and  audits  of  our  confor- 
mAnce  to  our  established  quality  control  procedures.  These  audits  act 
as  a  valuable  check  to  insure  that  our  quality  control  program  functions 
continuously  to  guarantee  the  high  quality  of  the  missile  systems. 

When  the  missile  is  brought  into  the  final  checkout  area,  for  final 
Acceptance  Tests,. pneumatic  lines  which  connect  the  missile  and  the 
ground  air  supply  are  hooked  up.  Through  these  lines  the  air  storage 
Davteries  or  spheres  as  they  are  called  are  pressurizea.  This  pneumatic 
pressure  is  used  during  flight  to  operate  valvas,  float  the  gyros,  for 
astxtuae  control  of  the  missile,  and  pressurization  of  the  fuel  tanks. 
Electrical  harnesses  connecting  the  ground  consoles  with  the  missile  are 
hooked  up.  It  is  through  the  use  of  the  switches  ,nd  the  meters  on  the 
ground  consoles  that  the  tests  are  performed.  Other  equipment  is  con¬ 
nected,  such  as  interrupter  boxes,  dummy  plugs,  interrupter  harnesses, 
gauges,  anu  meters  which  simulate  various  conditions  that  would  occur 
during  flight  and  monitoi  pressure,  current,  and  Voltage  at  the  various 
points  within  the  missile  system.  For  example,  due  to  the  safety  re¬ 
quirements  the  fuel  and  oxidizer  tanks  are  not  pressurized  in  this  area, 
actual  ignition  ana  nainstage  burning  cannot  be  accomplished,  launching 
ring  lift-off  is  not  accomplished,  but  must  all  be  simulated.  These 
ore  accomplished  by  jumpering  certain  connections  in  miSBile  circuitry, 
installing  a  time  delay  relay  to  simulate  ignition  delay,  and  using  a 
pneumatically  operated  fixture  to  disconnect  the  electrical  connections 
at  the  tail  plugs  as  in  lift-off.  To  provide  error  signals  to  the  control 
system  the  stabilized  platform  is  installed  in  a  turn-tilt  stand  yhich 
can  deflect  the  stable  tablo  into  any  prescribed  orientation  and,  thus 
produce  corresponding  output  signals. 

The  circuitry  in  the  tent  equipment  is  designed  such  that  most  of 
the  components  aboard  the  missile  can  be  operated  individually.  For 
instance;  a  valve,  computer,  radar  beacon,  blower  motor,  etc.,  can  be 
turned  on  or  off  at  will  wit.iu'  t  going  through  automatic  launch  seq¬ 
uencing.  After  each  of  these  components  is  t  ested  individually  it  is  then 
tested  In  conjunction  with  the  other  components  and  interconnecting 
harnessing  within  its  system,  After  each  of  the  systems  have  been  checked 
for  proper  operation  ell  the  systems  are  operated  together  in  a  final 
simulated  flight  tost.  An  example  of  tills  progressive  testing  is  the 
operation  of  the  aiain  fuel  valve.  The  valve  would  be  tested  as  a  com¬ 
ponent  to  determine  that  it  functioned  correotly.  Then  it  would  oe 
operatod  as  part  of  the  automatic  launch  sequence  overall  tost  with  only 
the  propulsion  system  operated.  During  tho  simulated  flight  test  it 
would  be  chocked  again  in  conjunction  with  tho  guidance  system,  measuring 
system,  and  tracking  system.  Tho  simulated  flight  tost  would  determine 
that  them  wore  no  signals  from  ono  system  wtiich  would  interfere  with  any 
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system  by  continuous  surveillance#  When  necissary,  :  hoy  maj  ai  per¬ 
form  direct  inspection  of  the  supplier's  products  and  have  u!ut  a  hority 
to  accept*  reject  or  requestiwork  of  nonconfi.  rminb  material  proai  ,u<i 
by  the  supplier.  They  ulso  assist  our  suppliers  „ .  correcting  a  y 
quality  problem  areas.  By  assuring  the  performance  all  necese_xy 
inspections  anti  tests  at  the  supplier's  location,  l  .  is  thus  poss'ble 
to  reduce  the  amount  of  inspection  necess-irv  ’-:hc.  .  •  j  .  -tr  •>-»  -  oei -’d 

at  Missile  Di/jsico. 

The  bulk  of  the  parts  a  u  - ^mponents  which  we  purchase  are  procured 
from  smaller  vendors  and  subco  rectors  whero  the  volume  of  parts  pro¬ 
cured  makes  it  more  economical  .>  perform  the  inspection  and  acceptance 
right  at  Missile  Division.  These  items  are  thoroughly  inspected  and 
tested  upon  receipt  to  assure  that  all  quality  requirements  are  met. 

Tho  amount  of  inspection  performou  is  generally  covered  by  the  nature 
of  the  part,  as  well  as  the  past  quality  performance  of  the  supplier 
involved.  Even  where  inspection  acceptance  is  performed  at  Missile 
Division,  close  liaison  is  maintained  with  the  supplier  to  inform  him 
of  his  quality  performance  and  of  proolem  areas  requiring  correction. 

Wnsre  necessary,  both  the  supplier  liaison  people  and  tho  Quality  Con¬ 
trol  Engineers  work  closely  with  the  vendor  to  assist  him  in  iirproving 
his  Quality  Control  system  in  order  to  prevent  recurring  discrepancies. 

All  parts  fabricated  and  assembled  within  the  Missile  Plant  are 
inspected  and  tested  by  our  In-Process  Inspection  Section.  This  involves 
all  types  of  dimensional  and  mechanical  inspection,  pressure  testing  and 
electrical  functional  tests.  Inspection  is  performed  continuously  during 
the  fabrication  and  build-up  process.  During  the  assembly,  inspections 
are  performed  at  predetermined  points  where  a  high  possibility  of  error 
may  exist  in  order  to  completely  control  the  build-up  process.  At.  the 
completion  of  the  missile  assembly,  a  final  inspection  is  performed  on 
the  entire  missile  including  pressure  testing,  continuity  and  installation 
resistance  check  of  harnesses  and  optical  alighnment  inspection.  The 
missile  is  not  released  to  the  Final  Acceptance  Testing  or  Final  Checkout 
operation  until  the  completeness  and  acceptability  of  the  entire  assemoly 
has  been  verified. 

The  Final  Acceptance  Testing  is  responsible  for  performing  a  complete 
functional  system  test  of  the  entire  missile.  During  this  test,  the 
missile  is  put  through  the  complete  sequence  of  operations  normally 
performed  prior  to  launching  and  simulated  flight  test.  A  similar  type 
of  test  is  performed  on  a] 1  units  of  Ground  Support’  Equipment  before 
they  are  delivered.  The  majority  of  the  Acceptance  Testing  is  performed 
by  this  section. 

Another  very  important  phase  of  the  Quality  Control  Program  is 
carried  out  by  the  Quality  Assurance  Section  which  is  responsible  for 
continuously  monitoring  and  measuring  our  product  quality.  Quality  data, 
in  the  form  of  inspection  and  test  records,  andreports  of  defects  and 
failures  are  fed  into  our  Quality  Analysis  Group  from  subcontractors, 
vendors,  in-plant  inspection,  and  field  locations.  This  data  is  system¬ 
atically  categorized  and  stored  by  means  of  automatic  data  processing 
equipment  and  extracted  for  statistical  analysis.  This  data  represents 
a  complete  historical  quality  record  which  is  used  in  determining  quality 


other  of  the  systems.  For  example,  poor  installation  or  a  floating  ground  on 
one  of  the  connections  in  the  propulsion  system  might  allow  stray  current 
to  appear  in  the  Guidance  System  and  introduce  incorrect  control  move¬ 
ment.  During  the  test,  all  of  the  radio  frequency  gear  is  operated  to 
\isure  that  there  is  no  stray  noise  from  rotating  equipment  nor  harmon¬ 
ics  of  tracking  signals  to  interfere  with  the  telemeter  signals  nor  the 
operation  of  other. tracking  or  range  safety  equipment. 

The  simulated  flight  test  is  the  final  test  run  on  the  missile. 

After  the  test  results  are  thoroughly  evaluated  and  it  is  verified  that 
the  missile  performed  properly,  it  ie  then  released  to  be  packaged, 
weighed,  and  shipped.  At  this  time,  it  need  only  be  erected,  armed,  and 
fueled  incrder  to  be  ready  to  fire. 

r  h  D  Missiles  are  flown  to  Cape  Canaveral  for  firing.  Earlier  mis¬ 
siles  were  given  a  receiving  inspection  test  in  the  hangar  prior  to  erection 
on  the  pad.  However,  later  missiles  were  delivered  directly  to  the  pad 
for  the  final  mating  of  the  missile  to  the  firing  equipment  and  series 
of  tests  prior  to  launch. 

The  tactical  Jupiter  Missiles  are  mated  to  the  actual  Ground  Support 
Equipment  which  will  form  a  launch  complex  in  the  field.  These  mating 
tests  assure  the  compatibility  of  the  system  equipment  and  procedures  and 
point  out  any  problems  which  might  result  in  deployment.  After  mating, 
each  system  is  packaged  and  shipped  overseas. 

The  final  acceptance  testing  of  the  ground  support  equipment  follows 
the  same  criteria  as  the  missile  testing.  Each  vehicle  goes  through  a 
complete  pressure,  electrical  and  mechanical  test  prior  to  and  after  a 
lengthy  road  test.  This  insures  that  the  equipment  which  launches  a  mis¬ 
sile  in  the  field  will  be  reliable  as  the  missile  itself. 

Thus  the  final  sta^e  in  the  manufacture  of  a  Guided  Missile  System 
is  accomplished.  The  Final  Acceptance  Testing  is  the  final  buyofl  as  the 
government  accepts  the  equipment.  The  detailed  procedures  and  specifi¬ 
cations  used  in  this  area  have  certainly  made  a  great  contribution  toward 
the  proven  flight  reliability  of  both  the  Redstone  and  Jupiter  Systems. 

Acknowledgement  to  Mr.  I.  Katzman,  Chrysler  Corporation  Missile  Division 
for  portions  of  the  naterial  used  in  this  paper. 


WCFK  SMARTER,  NOT  HARDER  TO  ACHIEVE  RELIABILITY 
By  William  J.  Hill 

Manager,  Statistical  Services  Section 
Ford  Instrument  Company 
Division  of  Sperry  Rand  Corporation 

High  reliability  and  quality  requirements  are  synonymous  with 
tuiy  missile  program.  It  is  therefore  extremely  important  to  maintain 
control  over  manufacturing  and  assembly  processes  to  assure  that  tbe 
reliability  and  quality  conmitments  are  either  met  or  bettered.  Most 
companies  try  to  assure  meeting  these  commitments  by  standardizing 
inspection  instructions  (such  as  classification  of  characteristics), 
tightening  up  of  acceptable  quality  levels,  and  increasing  tho  number 
of  as  well  as  quality  of  their  measuring  equipment.  While  this  is 
rcenosary,  it  is  not  all  that  must  be  dore  to  assure  meeting  the  high 
quality  and  reliability  levels  expected  of  the  missile  industry. 

Basically,  we  have  found  four  areas  wherein  the  expenditure  of 
a  little  effort  on  the  part  of  quality  control  and  reliability 
organizations  pay  dividends  out  of  all  proportion  to  the  time  spent. 
These  four  areas  have  been  recognized  by  many  as  problem  areas.  The 
first  is  measurement  problems  of  correlation  and  repeatability  of 
equipment  and  the  standardization  and  definition  of  what  is  to  be 
measured.  This  is  perhaps  one  of  the  biggest  problems  for  any  quality 
control  organization.  It  becomes  increasingly  important  to  the 
company  and  the  government  as  we  subcontract  and  purchase  components 
and  sub-assemblies  from  two  or  more  suppliers.  - 

The  second  area  is  quality  reporting.  Quality  reporting  need 
not  be  expensive  but  must  define  quality  in  terms  that  management  can 
readily  understand.  As  quality  control  reports  quality,  or  lack  of  it, 
to  manufacturing  management  and  to  top  management,  it  is  essential  that 
they  have  a  feeling  of  what  poor  quality  or  poor  reliability  means. 

The  easiest  way  to  do  this  is  to  hit  them  in  the'pocketbook  by  reporting 
quality  in  terms  of  cost. 

The  third  area  is  the  j roblera  of  adequately  staffing  inspection 
and  test  operations  so  that  the  maximum  efficiency  is  obtained  from 
them.  Needless  to  say  many  of  you  have  experienced  the  situation  where 
throwing  extra  people  into  inspection  and  test  only  accomplished  a 
screening  operation  and  a  poor  one  at  that.  But  have  any  of  you  cut 
down  inspection  or  test  time  without  cutting  down  the  quality  or  relia¬ 
bility  of  your  product? 
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The  last  but  by  no  means  the  least  important  area  is  a  proper 
evaluation  of  changes  in  any  product  or  process  through  the  use  of 
a  properly  designed  statistical  study.  This  is  especially  important 
in  the  missile  area  where  you  have  small  production  runs  and  tre¬ 
mendously  high  costs.  A  gocf  here  costs  money.  It  also  slows  down 
production  and  may  lead  to  your  replacement  as  a  missile  contractor 
by  some  other  organization  which  ha3  a  more  efficient  method  of 
evaluating  changes  in  both  products  and  designs. 

How  that  we  have  highlighted  the  four  major  areas,  let's  examine 
each  one  in  detail  to  see  what  is  necessary  to  obtain  the  maximum 
quality  and  reliability  with  your  present  organization  by  working 
smarter,  not  harder. 


Measurement  Problems 

We  have  found  from  our  experience  with  vendors  and  our  own 
internal  experience  that  there  are  three  basic  problems  associated 
with  measuring  equipment.  The  first  is  the  definition  of  what  is  to 
be  measured.  While  this  may  sound  extremely  trivial,  nevertheless 
when  you  send  a  vendor  a  blueprint  and  tell  him  you  want  a  certain 
dimension,  you  may  run  into  more  difficulties  than  you  expect.  This 
is  an  especially  big  problem  in  areas  where  items  are  being  measured 
in  millionths  of  an  inch  such  as  air  bearing  gyros.  In  order  uo  solve 
this  it  is  necessary  that  you  spell  out,  for  critical  dimensions, 
exactly  what  you  want  measured,  how  you  want  it  measured,  the  type  of 
equipment  to  be  used,  the  number  of  measurements  to  be  taken,'  and 
where  they  should  be  taken.  Once  you  have  done  this  the  question  then 
arises,  does  your  basic  program  of  measurement  control  really  give  the 
necessary  protection?  To  ascertain  the  effectiveness  of  your  control 
program,  the  following  questions  must  be  answeredJ 

(1)  What  is  the  precision  or  repeatability  of  the  measuring 
equipment  in  U3e? 

(2)  Do  different  operators  using  the  same  measuring  equipment 
obtain  the  same  readings? 

(3)  How  many  of  the  disagreements  between  in-process  and  final 
inspection,  between  incoming  and  supplier's  final  inspection, 
and  between  final  inspection  and  purchaser's  inspection  are 
the  result  of  poor  repeatability  or  lack  of  correlation  of 
equipment? 

Measuring  equipment  graduated  in  ten- thousandths  of  an  inch,  often 
is  incapable  of  repeating  within  one  graduation.  In  addition,  readings 
taken  on  the  same  part  with  two  "identical"  measuring  machines  often 
differ  by  more  than  one  graduati >n.  Therefore,  along  with  a  basic 
control  program,  a  manufacturing  organization  must  conduct  continuing 
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studies  of  the  correlation  and  repeatability  of  measuring  equipment 
which  they  and  t.heir  suppliers  use.  Also,  suitable  inspection 
programs  must  be  devised  which  uill  insure  that  tolerances  will  be 
held  should  the  repeatability  of  measuring  equipment  be  poor  and 
which  cannot  be  improved  by  adjustment  or  recalibration. 


Aft.Eftffnp.lfl 

Shop-produced  components  and  purchased  "components  of  a  gyro 
assembly  did  not  fit  together  properly  because  of  the  differences 
in  measuring  equipment  and  operating  personnel  at  the  two  sources  of 
manufacture,  even  though  each  was  maintaining  tolerances  within  its 
own  operations.  The  supplier  of  purchased  material  graded  bearings 
(for  inside  and  outside  diameter)  into  four  classes  -  A,  B,  C,  and 
C  -  for  each  dimension.  Each  class  had  a  50  millionths  inch  spread 
in  dimension  and  each  bearing  was  identified  with  two  letters:  AA, 

AD,  BC,  and  so  on.  At  assembly  of  the  gyro,  the  bearings  were  matched 
by  their  code  letter  to  shop-manufactured  parts.  Parts  which  went 
together  on  paper  did  not  on  the  assembly  bench.  When  shop  supplied 
parts  were  reinspected,  the  same  measurements  were  obtained  as  before, 
however,  bearings  labeled  AA  were  found  upon  reinspection  to  be  BA, 

AB,  or  BB. 

The  supplier  graded  each  bearing,  put  it  in  a  heat-sealed  bag, 
and  then  into  a  box  marked  with  the  grade.  To  check  grading  each  bag 
had  to  be  opened,  the  bearing  measured,  cleaned  and  relubricated,  and 
repacked  in  another  heat-sealed  bag.  A  risk  of  contamination  occurred 
because  of  this.  Still  another  problem  arose  -  the  frequency  distri¬ 
bution  of  bearings  from  our  inspection  was  not  the  same  as  from  the 
supplier.  This  meant  that  the  supply  of  a  particular  bearing  grade 
needed  for  assembly  was  insufficient. 


„v 


Cur  approach  to  this  problem  was  to  determine  the  repeatability 
of  both  shop  and  supplier  measuring  equipment,  the  average  difference  L 

in  measurement  between  us,  and  from  this  the  correction  factor  which 
would  enable  the  supplier  to  correctly  grade  readings  to  shop  measure-  V 

ments.  An  experiment  was  designed  to  enable  measurement  differences 
to  be  properly  assigned  to  the  following: 


(1)  Different  measuring  machines  used  i 

(2)  Different  setting  rings  used  / 

(3)  Different  operators  (two  from  the  supplier  and  two  from  our  shop) 


Four  different  size  bearings  were  used,  one  for  each  class.  It  was 
assumed  that  a  significant  difference  would  be  found  between  them.  If 
no  significant  difference  were  found,  there  would  be  no  sense  to  grading. 
To  determine  the  precision  of  the  measurements,  it  was  necessary  to 
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Include  a  repeatability  study  in  the  experiment.  In  the  repeatability 
study  each  test  condition  was  repeated  at  random  times  during  the 
course  of  the  experiment.  This  gave  two  complete  sets  of  data  for  the 
experiment.  Analysis  of  the  difference  between  these  two  sets  of  data 
peiiiitted  determination  of  repeatability.  The  results  of  this  experi¬ 
ment  for  the  inside  diameter  measurements  are  in  Figure  1. 

Similar  data  were  obtained  for  the  outside  diameter  measurements. 
The  analysis  for  both  sets  of  data  was  performed  in  the  same  manner. 
There  were  two  stops  involved  in  this  analysis:  determining  the  repeat¬ 
ability  of  measurements  and  determining  significant  causes  of  measure¬ 
ment  difference. 


Determining  Repeatability 

Repeatability  was  determined  by  taking  the  difference  or  range 
between  measurements  obtained  under  identical  conditions.  The  average 
range  over  all  measurements  constituted  our  estimate  of  repeatability. 

Formal  statistical  techniques  assume  that  repeatability  is 
relatively  constant  throughout  an  experiment.  In  this  case  this 
assumption  could  be  verified  by  using  a  range  control  chart  where 
95^  of  the  individual  ranges  should  fall  in  the  interval  from  0  to 
2.4 8  H,  where  H  was  the  average  range.  For  the  values  shown  in  Figure  1 
the  interval  was  0  to  28.8.  The  ranges  were  plotted  on  a  control  chart, 
Figure  2.  This  chart  showed  that  all  the  points  were  within  the  control 
limits.  However,  repeatability  of  the  first  sixteen  points  (our  equip¬ 
ment)  appeared  to  be  much  better  than  the  repeatability  of  the  second 
sixteen  points  (supplier  equipment).  When  the  average  repeatabilities 
of  the  machines  were  compared,  by  a  variance  ratio  test,  a  significant 
difference  in  repeatability  was  found.  Further  analysis  showed  that 
there  was  no  significant  difference  in  repeatability  between  the  two 
shop  operators  or  the  two  supplier  operators.  Based  on  this  variance 
ratio  test,  it  was  concluded  that  the  repeatability  of  the  supplier's 
equipment  must  improve  if  bearings  vrere  to  be  graded  properly.  This  is 
true,  of  course,  even  if  no  other  differences  existed  between  our  measure¬ 
ments  and  those  made  by  tne  supplier. 


Significant  Causes  of  Differences 

To  determine  whether  significantly  different  measurements  were 
obtained  from  the  different  measuring  equipment,  different  operators  or 
different  setting  rings,  the  average  value  of  the  measurements  under  each 
condition  was  compared  to  the  average  measurement  for  the  experiment  by 
the  use  of  a  control  chart. 
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The  extent  of  random  variation  of  the  average  measurement  from 
the  over-all  average  should  be  within  limits  calculated  as  follous: 
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Where  U  is  upper  control  limit,  or  XL,  L  is  the  lower  control  limit, 
or  LCL.  TH  is  the  grand  average  -  from  Figure  2,  t.  is  Student's  "t", 

X  is  the  average  of  «n  the  measurements,  and  n  is  the  number  of  readinge 
'.rod  to  obtain  the  average  measurement  for  a  level  of  a  variable.  For 
the  data  shown  there  were  two  sets  of  limits,  one  set  for  the  variables 
with  two  levels  (machine,  satting  rings)  and  one  set  for  the  variables 
with  four  levels  (bearings,  operators) .  By  plotting  these  limits  a 
control  chart  was  obtained  for  main  effects  (Figure  3).  A  common  grand 
average  was  used  for  determining  the  upper  and  lower  control  limits  for 
the  machine  setting  ring,  and  bearing  effects.  However,  as  the  operator 
effect  was  nested  within  the  machine  effect,  operator  1  was  compared 
against  operator  2  for  both  the  shop  and  the  supplier  by  using  the  average 
measurement  obtained  at  the  shop  and  the  supplier  respectively.  The 
control  limits  around  these  averuges  are  the  same  distance  from  the 
average  as  the  control  limits  around  the  bearing  effect. 

If  a  point  falls  outside  the  control  limits,  it  indicates  that  the 
effect  of  different  levels  of  tnat  variable  cannot  be  attributed  to 
experimental  error  (repeatability).  This  effect  is  then  said  to  be 
significant.  On  the  other  hand,  if  no  point  falls  outside  the  control 
limits,  the  difference  in  levels  of  that  variable  is  due  to  experimental 
error  (repeatability)  and  the  effect  of  that  variable  is  not  significant. 
Figure  3  shoued  that  the  effects  of  the  machine  and  bearing  were  signi¬ 
ficant.  ho  interactions  were  significant. 

Two  methods  to  improve  grading  procedure  were  recommended  to  the 
supplier.  Either  the  some  type  of  measuring  equipment  should  be  used 
by  him  as  that  used  in  our  shop,  or  the  basic  difference  between  shop 
measurements  could  be  subtracted  from  his  measurements  before  grading 
the  bearings.  In  addition,  it  was  recommended  that  the  supplier  use 
the  average  of  two  measurements  for  those  bearings  whose  initial  measure¬ 
ment  was  within  twenty  millionths  of  the  grading  tolerance. 

Since  our  first  step  in  this  direction  we  have  expanded  our  study 
of  measurement  problems  to  cover  all  critical  material.  We  have  found 
many  instances  where  there  was  a  lack  of  correlation  between  our 
measuring  equipment  and  our  vendor's  measuring  equipment.  We  have  also 
found  instances  where  excellent  correlation  existed  between  our  measuring 
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equipment  but  because  of  poor  repeatability  we  rejected  good  parts 
or  accepted  bad  ones.  To  remedy  this  situation  we  have  conducted 
correlation  and  repeatability  studies  with  our  vendors  and  have 
effected  corrective  action  wherever  necessary.. 

uur  Test  Department  has  recently  initiated  a  program  whereby 
repeatability  checks  will  be  performed  by  the  calibration  section. 

As  most  test  equipment  is  expensive,  any  equipment  that  does  not  have 
sufficient  repeatability  to  distinguish  between  acceptable  and  reject- 
able  material  will  be  used  for  the  measurement  of  less  critical 
material,  wherever  possible.  If  this  is  not  possible  and  if  we  cannot 
obtain  replacement  equipment,  we  resort  to  selective  double  inspection, 
ihio  is  a  procedure  whereby  parts  whose  measurements  are  within  plus 
or  minus  II  (the  average  repeatability  of  the  measuring  equipnent)  from 
the  tolerance  limit  will  be  inspected  twice.  The  average  of  these 
two  measurements  is  then  used  as  the  criterion  for  acceptance  or  rejection. 


Quality  Reporting 

One  of  the  biggest  problems  faced  by  a  quality  control  organization 
is  how  to  successfully  report  quality.  As  most  quality  reports  deal 
with  past  history,  it  is  obvious  that  we  should  have  a  rational  way  of 
determining  the  quality  level  experienced.  Many  people  have  their  own 
schemes  for  reporting  quality.  Some  base  it  on  the  percent  defective 
and  others  on  the  defects  per  unit  of  the  inspected  material.  These 
methods,  while  reporting  one  measure  of  the  quality  level,  do  not  convey 
it  in  terms  that  top  management  understands.  If  an  item  or  a  department 
or  a  plant  has  a  quality  level  of  1%  defective,  does  this  cost  the 
company  more  than  one  with  a  quality  level  of  5%  defective?  Management 
is  paying  for  a  quality  product,  therefore,  they  are  interested  in 
knowing  how  much  of  this  payment  is  lost  by  the  production  of  poor 
quality  material.  Thus,  one  of  the  first  characteristics  of  good  quality 
reporting  is  to  report  the  quality  level  in  terms  that  management  can 
understand  and  that  can  be  used  by  shop  management  to  obtain  better 
quality. 

The  second  characteristic  is  that  the  quality  information  should 
be  based  upon  records  that  are  currently  available.  Almost  all  plants 
have  a  Cost  Accounting  Department  where  you  can  obtain  realistic  figures 
of  the  time  spent  on  a  given  job  as  well  as  the  time  required  to  rework 
the  job  because  of  poor  quality.  The  cost  of  scrapping  material  is  also 
determined  by  most  Cost  Accounting  Departments.  This  is  all  the  basic 
information  you  need,  h'ost  of  this  information  is  reported  to  management 
each  month,  but  this  reporting  does  not  include  any  analysis  that  is 
immediately  useful  to  detect  quality  trends. 

The  third  characteristic  any  effective  quality  reporting  system 
should  have  is  that  it  must  be  inflation-proof.  If  we  report  our  quality 
in  terms  of  dollars  and  cents,  we  can  expect  that  the  steady  upward  creep 
of  inflation  will  render  invalid  any  long  term  quality  comparisons. 
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However,  if  it  takes  two  hours  to  make  a  part  using  a  particular  setup 
today,  it  will  take  two  hours  to  produce  that  same  part  using  the  same 
setup  two  years  from  now.  Therefore,  the  hours  spent  to  produce,  rework, 
or  the  hours  lost  in  scrapping  a_given  part  or  assembly  are  a  most 
effective  measure  of  quality. 

How  how  is  this  information  used?  Figure  4  shows  Ford's  Manufac¬ 
turing  Quality  Level  Report,  This  report  has  been  in  use  for  over  one 
year  and  has  been  well  received  by  management,  by  our  prime  contractors, 
and  by  the  government  representatives  stationed  at  our  plant.  The 
report  is  simple,  straightforward,  and  takes  a  minimum  of  time  to  prepare. 
If  you  will  look  at  any  given  block  on  the  report,  you  will  see  that  we 
report  three  figures  -  scrap  hours,  rework  hours,  and  production  hours. 
These  figures  are  all  obtained  from  Cost  Accounting.  Our  measure  of 
quality  is 

M.Q.L.  =  Scrap  +  Rework _ 

Scrap  +  Rework  +  Production 

This  is  the  Manufacturing  Quality  Level. 

The  rework  hours  are  charged  to  the  department  that  caused  the 
rework  and  do  not  necessarily  reflect  on  the  department  that  did  the 
rework.  Thus,  if  a  high  percentage  of  rework  is  caused  by  drilling,  it 
may  be  that  the  Bench  Mechanics  Department  spent  a  large  number  of  hours 
removing  broken  drill  bits.  Nevertheless,  the  hours  spent  by  Bench 
Mechanics  on  drilling  rework  are  charged  to  the  Drilling  Department. 

The  production  hours  for  drilling  and  jig  boring  added  together 
give  the  production  hours  for  the  Drilling  Department.  The  same  applies 
for  the  scrap  and  rework  hours.  Time  lost  in  scrapping  or  reworking 
items  for  which  no  one  department  can  be  hold  responsible  is  entered 
in  the  shop  total  slot  for  either  scrap  or  rework  hours.  Thus,  the 
figures  shown  in  the  scrap  and  rework  slots  of  the  shop  total  box  will 
be  greater  than  the  sum  of  all  the  scrap  an<J  rework  hours  shown  for  the 
four  departments  underneath  it. 

This  report  is  used  by  almost  all  divisions  of  the  company.  The 
shop  manager  receives  copies  of  the  rework  cards  and  scrap  tickets  vhich 
indicate,  wherever  possib'e,  the  employee  responsible  for  producing  the 
poor  quality  material.  The  shop  manager's  office  maintains  a  record  of 
the  amount  of  rework  and  scrap  produced  by  each  employee.  This  is  used 
by  shop  management  to  rate  the  employees,  to  determine  if  additional 
training  is  needed  by  an  employee,  and  to  warn  the  employee  when  excessive 
quality  failures  have  been  charged  against  him.  Furthermore,  this  report 
is  used  by  the  shop  manager  to  rate  the  effectiveness  of  hi3  managers  and 
foremen  in  maintaining  the  quality  of  their  production  output. 

Quality  Control  uses  thin  report  to  assist  them  in  determining  causes 
of  repetitive  defects.  Consistent  rejects  which  require  either  rework 
or  scrap  are  analysed  to  determine  whether  additional  in-process  checks 
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or  an  increased  overage  la  required  (provided  there  axe  no  other 
possible  corrective  actions  which  will  remedy  thia  condition).  This 
report  also  assists  Quality  Control  in  assigning  inspection  coverage 
to  areas  that  have  3hown  a  consistently  poor  quality  level.  The 
policy  is  to  assign  the  inspectors  to  in-process  work  so  as  to  avoid  the 
production  of  scrap  or  rework  material  rather  than  to  increase  the  end 
of  the  line  inspection  which  merely  sorts  the  good  from  the  bad. 

In  order  to  inform  the  shop  personnel  of  their  quality  performance 
a  Quality  Level  Chart  is  posted  in  each  foreman's  area  depicting  the 
quality  results  for  a  given  month.  (See  Figure  5.)  The  lower  part  of 
this  chart  shows  the  desired  quality  lovel  for  the  particular  area* 

The  total  height  of  each  line  on  the  chart  indicates  the  actual  quality 
level  produced  and  is  the  same  as  that  shown  on  the  Manufacturing  Quality 
Level  Report.  When  these  charts  W6ie  started  almost  every  employee 
spent  some  time  looking  at  them  and  asking  his  foreman  questions  about 
them.  The  foremen  had  been  briefed  and  were  able  to  supply  the  shop 
personnel  with  answers  that  they  could  understand. 

The  benefits  of  this  report  to  the  con^any  were  obvious.  It 
supplied  realistic  quality  information  that  could  be  used  by  all 
divisions  at  a  very  nominal  cost.  Since  its  inception,  we  have  noticed 
a  consistent  improvement  in  tho  quality  level  of  the  shop  as  a  whole  as 
well  as  in  many  of  the  individual  area  ~  —  - ‘  .1 1 *  wiu  ii w «*.  con o ■ 

therefore,  is  that  this  report  has  boen  boneficial  to  us  and  would  be 
beneficial  to  you. 


Adequate  Inspection  and  Test  Coverage 

The  question  of  adequate  inspection  and  test  coverage  is  one  that 
cannot  be  merely  decided  by  rule  of  thumb  such  as  the  practice  of 
utilizing  one  inspector  for  every  seven  operators  in  the  shop  or  one 
tester  for  every  five  assemblers  regardless  of  the  type  of  work  per¬ 
formed.  In  doing  this  we  run  one  of  two  risks,  either  we  will  have 
insufficient  coverage  so  that  our  reliability  and  quality  levels  will 
deteriorate  or  we  will  be  paying  too  much  to  obtain  a  very  small  increase 
in  our  quality  and  reliability  levels. 

Rather  than  use  a  rule  of  thumb,  we  decided  to  measure  the  efficiency 
of  inspection  and  tost  operations  in  various  arca3.  The  results  of  this 
arc  shown  in  the  attacned  chart  (Figure  6)  which  plots  quality  efficiency 
versus  the  ratio  of  inspectors  and  testers  to  manufacturing  personnel. 

T'rcm  tins  you  will  note  that  ue  have  never  achieved  IOC#  efficiency.  If 
any  of  you  claim  to  have  achieved  it,  you  are  kidding  yourself.  You 
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may  have  come  close  to  it  but  in  this  business  a  miss  is  still 
expensive.  From  this  chart  you  see  that  it  is  relatively  easy, 
on  paper,  to  obtain  better  than  a  90$  inspection  efficiency. 

However,  to  achieve  99$  or  99.9$  inspection  efficiency  would  be 
extremely  difficult.  Even  using  a  1  to  1  ratio  the  best  efficiency 
was  approximately  96$.  This  can  be  attributed  to  the  fact  that 
many  inspection  and  test  operations  are  extremely  tedious  ind 
require  the  evaluation  of  hundreds  or  even  thousands  of  ci  eck 
points  where  the  decision  to  accept  or  reject  is  based  upon  the 
inspector's  judgment.  One  inspector  may  say  it  is  good  and  another 
may  say  it  is  bad,  etc. 

Now  how  can  this  be  overcome  so  as  to  increase  our  inspection 
and  test  efficiency  without  bankrupting  the  organization.  There 
are  two  ways  of  doing  this.  The  first  and  most  important  is  to  assure 
that  classification  of  characteristics  or  inspection  instructions 
spell  out  in  detail  exactly  what  you  will  accept  and  what  you  will 
not  accept.  The  realm  of  all  grey  area  decisions  should,  wherever 
possible,  be  removed  from  the  hands  of  inspectors  and  placed  in  the 
hands  of  the  supervisors.  It  is  far  easier  to  control  the  supervisors 
than  the  inspectors  and  testers  in  a  given  plant.  Thi3  is  the  first 
step.  Without  doing  this,  the  second  step  may  not  produce  tiny  results 
at  all.  The  second  step  involves  selecting  the  best  combination  of 
inspection  to  production  ratios  and  following  thi3  up  with  an  audit 
inspection  to  evaluate  the  efficiency  of  your  inspection  and  test 
organizations.  The  cost  of  tliis  can  be  plotted  as  a  function  of  your 
total  test  organization  or  your  increase  in  efficiency  and  an  appro¬ 
priate  cutoff  point  selected  by  management.  The  combination  of  these 
two  will  give  you  the  optimum  coverage  in  any  one  inspection  and  test 
area.  A  third  item  which  should  be  looked  into  is  the  use  of  automatic 
testing  equipment  such  as  attribute  measuring  devices  instead  of 
variable  type  devices.  Again,  the  cost  of  these  items  should  be 
balanced  out  against  the  increased  inspection  and  tesfefficiency. 

The  above  three  items  can  effectively  be  handled  by  Quality  Control 
or  reliability  organization  within  your  company. 

Cutting  Inspection  Costs 

Based  or.  the  previous  discu.-sion,  you  will  probably  find  that 
there  are  many  areas  which  require  increased  inspection  coverage.  If 
you  look  through  the  records  you  maintain,  you  will  also  find  that 
there  are  many  areas  where  inspection  costs  can  be  cut.  We  have  found 
parts  that  take  three  hours  to  manufacture  may  take  ten  hours  to 
inspect.  Sounds  ridiculous  doesn't  it,  yet  it  happens.  If  nothing 
is  done  about  it,  you  noon  find  that  you  are  priced  out  of  business. 

Yet  you  cannot  dimply  eliminate  inspection.  To  do  so  would  be  sheer 
folly  for  you  would  also  be  out  of  business  because  of  the  poor 
quality  produced.  To  solve  this  problem,  namely  bringing  the  inspec¬ 
tion  time  in  line  with  the  shop  time  as  well  as  maintaining  exacting 
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quality  requirements ,  we  reviewed  inspection-  records  by  a  technique 
known  as  error  analysis  and  came  to  the  conclusion  that  inspection 
costs  could,  be  reduced  by  using  the  right  amount  of  inspection  at 
the  right  place  and  at  the  right  time.  Let  us  look  at  a  typical 
example  where  error  analysis  has  been  applied  and  see  the  benefits 
that  have  accrued  from  it. 

Cam  Cost  Cut 

Ford  Instrument  Company  produces  various  types  of  cams  -  two 
dimensional  and  three  dimensional,  empirical  function  cams  and 
mathematical  function  cams,  cams  that  have  200  inspection  pointB  and 
cams  that  have  14,000  inspection  points.  The  majority  of  our  cams 
are  u3cd  in  analog  computers  produced  by  Ford  and  by  many  other 
companies.  V/e  pride  ourselves  on  the  quality  of  our  cams.  For  a 
while,  however,  our  pride  was  dimmed  somewhat  by  the  ever  increasing 
cost  of  our  cams.  When  we  manufactured  cams  on  the  job  shop  basis, 
100%  inspection  of  all  points  on  a  cam  was  considered  a  routine 
operation.  However,  when  we^received  contracts  for  volume  production 
of  cams  we  v/ere  faced  with  a  problem.  We  had  to  cut  our  cam  inspec¬ 
tion  cost  or  else  loose  the  business.  In  Figure  7  we  shov;  you  a 
typical  cam  produced  by  Ford.  It  is  a  magnetic  variation  cam  which 
feeds  correction  factor  into  a  compass  to  assure  that  the  proper 
magnetic  variation  is  added  to  or  subtracted  from  the  heading  so  that 
the  pilot  always  has  a  true  magnetic  heading.  This  cam  contains 
approximately  3,000  check  points.  These  3,000  points  cover  the 
magnetic  variation  between  the  latitudes  of  70°  north  and  70°  south. 
As  this  is  not  a  simple  cam  based  on  a  mathematical  function  but 
based  upon  data  supplied  by  the  U.S.  Coast  and  Geodetic  Survey,  the 
surface  of  the  cam  is  very  irregular.  For  this  reason  it  was  felt 
that  100%  inspection  was  an  absolute  must.  However,  we  soon  found 
out  that  the  inspection  records  for  this  cam  showed  a  remarkably 
consistent  pattern.  In  Figure  8  we  present  the  average  readings  and 
■pread  for  the  different  points  on  this  cam.  You  will  note  that  we 
seem  to  have  peaks  and  valleys  in  the  circled  areas.  This  pattern 
consistently  repeated  itself  from  cam  to  cam.  Using  a  well  known 
principle  of  Pareto  that  a  very  few  points  contribute  most  of  the 
variation,  we  analyzed  the  cam  and  found  that  approximately  36  points 
had  deviations  large  enough  to  warrant  continual  inspection.  We 
instituted  a  sampling  plan  using  these  36  points  (Figure  9) .  However, 
to  make  sure  that  the  quality  was  maintained  we  not  only  plotted  the 
results  of  inspection  of  these  36  points  but  periodically  audited  a 
cam,  that  is  did  100%  inspection  on  it.  The  results  of  this  100% 
inspection  verified  the  adequacy  of  the  plan.  It  was  felt  that  we 
could  reduce  our  sample  size  from  each  cam  to  every  third  cam.  Doing 
this  maintained  our  protection  and  at  the  same  time  permitted  a 
minimum  of  inspection. 
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In  terms  of  cost  savings  the  cam  inspection  procedure  was 
phenomenally  successful.  For  a  modest  investment  in  time  (approx¬ 
imately  40  hours  for  the  initial  analysis)  a  saving  of  31  hours  por 
cam  was  possible.  On  another  cam  which  contained  14,000  i».spection 
Points,  which  you  can  well  realize  in  not  only  a  large  number  of 
inspection  points  but  would  be  an  exceedingly  tedious  job  for  an 
inspector  to  go  through,  the  Bavings  were  far  greater.  This  tech¬ 
nique  has  been  extended  to  all  items  manufactured  at  Ford  Instrument 
Company.  Furthermore,  we  have  never  received  an  unfavorable  comment 
from  any  of  our  customers  or  from  our  own  assembly  areas.  This 
proves  that  inspection  costs  can  be  cut  while  still  upholding  quality 
and  reliability. 


The  Proper  Evaluation  of  Product  Chances 

The  proper  evaluation  of  product  changes  involves  many  divisions 
within  the  company.  In  ordor  to  properly  evaluate  changes  in  products 
or  processes  the  sound  application  of  some  common  statistical  tech¬ 
niques  is  required.  The  two  most  useful  are  the  design  and  analysis 
of  experiments  and  the  correlation  study.  These  statistical  studies 
are  not  the  type  that  you  can  tell  the  Engineering  or  Manufacturing 
Divisions  that  they  must  use.  You  must  first  sell  them  on  why  they 
should  use  them.  The  biggest  selling  point  for  their  use  is  that 
you  can  investigate  many  variables  with  a  minimum  number  of  teats. 

A  Review  of  Experimental  Designs 

An  experiment  is  designed  to  determine  if  there  is  any  signi¬ 
ficant  change  in  output,  performance,  yield,  etc.,  as  we  go  from  one 
level  of  a  variable  to  a  second  level  of  that-  variable.  Most  of  the 
time  tho  engineer  iu  interested  in  the  effect  upon  a  procost  or  an 
assembly  of  a  variable  ucting  by  itself.  However,  the  engineer  must 
also  be  aware  of  the  feet  that  variation  in  an  experimental  setup 
may  be  due  to  the  interaction  between  two  or  more  variables.  (Inter¬ 
actions  between  three  or  more  variables  are  rare  and,  furthermore, 
are  usually  hard  to  explain.)  The  predominant  interaction  that  we 
must  worry  about  is  that  between  two  variabloe. 

Therefore,  the  basic  problem  in  tho  design  of  an  experiment  1b 
to  be  able  to  determine  which  of  the  many  effects  aro  significant  and 
which  two  variable  interactions  are  significant.  How  what  type  of 
experiment  is  bent  suited  for  this?  There  is  no  one  best  experiment 
that  will  serve  uil  j.  ir-.cscs.  However,  if  you  confine  yoursolf  to 
two  levels  of  each  vari  ’’'le,  the  fractional  factorial  experiment  yields 
a  maximum  amount  of  inf  ■  T.a Lion  for  a  minimum  amount  of  testing. 

In  Figure  10  we  illustrate  u  "master"  fractional  factorial  experiment. 
If  we  use  variables  A  through  L’,  we  can  measure  their  offoct  plun  any 
interaction  between  the-",  by  using  the  first  eight  toots.  If  wc  have 
eight  variables  to  investigate,  wo  will  use  letters  A  through  II  and 
tests  one  through  sixteen.  Again,  wo  will  bo  able  to  measure  tho 
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effect  of  each  variable  as  well  aa  the  effect  of  different  inter¬ 
action  groups  of  two  variables.  If  we  have  sixteen  variables,  we 
will  use  all  tests  shown  in  this  figure  to  achieve  the  same  results. 

It  has  been  our  experience  that  we  seldom  encountered  experiments 
that  required  an  evaluation  of  more  than  sixteen  variables  at  one 
time,  therefore,  we  have  limited  the  set  of  experiments  shown  in 
Figure  10, 

Now  what  about  using  the  Latin  or  Graeco-Latin  square  type 
experiments.  If  you  stop  to  think  for  a  second,  you  will  realize 
that  the  Latin  and  Graeco-Latin  squares,  which  are  merely  special 
cases  of  a  fractional  factorial,  would  not  serve  our  purpose.  A a 
we  increase  tha  number  of  variables,  we  also  increase  the  number  of 
levels  of  each  variable,  thereby,  showing  a  rapid  increase  in  the 
total  number  of  tests  required  tc.  measure  the  effects  under  investi¬ 
gation,  Furthermore,  as  we  increase  the  number  of  levels,  we 
encounter  problems  in  trying  to  obtain  these  levels  for  each  variable. 
It  would  be  possible  to  design  fractional  factorial  experiments  in 
which  we  can  measure  the  effect  of  seven  variables  in  eight  tests, 
fifteen  variables  in  sixteen  tests,  thirty-one  variables  in  thirty- 
two  tests  and  so  forth.  However,  by  doing  this  we  would  only  be  able 
to  measure  the  effoct  of  the  variable  acting  by  itself  (assuming 
all  interactions  are  nonexistent) .  We  would  lose  all  information  on 
interactions.  As  interactions  can  be  important  in  many  of  our  engi¬ 
neering  applications,  it  is  felt  that  it  is  best  not  to  run  the  risk 
of  having  a  significant  interaction  escape  our  detection. 

With  f'c  above  background  in  mind,  we  designed  a  oeriefc  of 
fractlona  factorial  experiments,  whore  each  variable  is  used  at  two 
levels,  enabling  the  engineor  to  determine  which  main  effect  and  which 
of  several  interaction  groups  are  significant.  The  table  in  Figure  11 
contains  the  best  comoination  of  experiments  for  any  given  number  of 
variables  (up  to  sixteen).  Through  the  use  of  experiments  contained  in 
this  table,  the  engineer  con  obtain  the  maximum  amount  of  information 
with  a  minimum  amount  of  testing.  Ho  will  be  able  to  detect  any  main 
effects  that  are  significant.  If  he  should  find  any  interaction  group 
significant,  he  can  either  determine  why  only  one  interaction  in  that 
group  could  logically  bo  significant  or  perform  additional  tests  using 
this  first  experiment  is  part  of  a  larger  experiment  to  pick  out  which 
interaction  in  the  group  was  significant. 

The  column  labeled  Interaction  Group  in  Figure  11  is  used  as 
follows:  The  engineer  selects  the  underlined  interaction  for  analysis. 
If  this  interaction  is  not  significant,  this  means  the  remaining 
interactions  in  the  bracket  arc  also  nonsignificant  because  they  are 
confounded  with  each  other  and  with  the  underlined  interaction. 

However,  should  thin  interaction  be  significant,  thon  the  onginoor 
must  ask  the  following,  question,  which  one  of  those  interactions  io 
really  significant'.-'  As  they  arc  all  confounded  with  oach  othor,  wo 
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rely  upon  the  use  of  engineering  judgment  as  well  as  statistical 
evidence  at  hand.  How  is  this  done?  Let  us  assume  that  inter¬ 
action  AB  was  found  significant  in  the  sixteen  variable  experiment. 

The  engineer  may  have  reason  to  believe  that  interaction  CD,  EF, 
and  GH  are  not  possible.  Therefore,  this  would  leave  us  with 
interaction  AB,  JK,  and  LM  to  evaluate.  By  seeing  what  tests  have 
been  previously  run  on  this  interaction,  he  would  then  know  what 
additional  tests  must  be  run  to  obtain  a  measure  of  their  effect. 
Another  factor  to  consider  in  the  analysis  is  that  an  interaction 
can  be  statistically  significant  but  have  no  engineering  significance. 
This  occurs  when  the  points  of  an  interaction  fall  outside  the 
statistical  limits  but  are  still  inside  the  over-all  tolerance 
limits.  When  this  happens,  we  need  not  worry  about  the  significance 
of  the  interaction. 

In  order  to  determine  the  significance  of  the  main  and  inter¬ 
action  effects,  it  is  necessary  to  repeat  some  tests.  The  column 
labeled  Repeat  Tests  tells  which  tests  should  be  re-run  to  obtain 
this  measure  of  experimental  error.  Thus,  If  we  axe  running  an 
experiment  with  sixteen  variables,  we  would  want  to  repeat  eight 
of  the  thirty- two  tests.  These  oight  tests  should  be  re-run  at 
random  times  during  the  course  of  our  experiment.  The  difference 
between  readings  on  these  repeated  tests  is  our  measure  of  the 
experimental  error  and  is  used  to  determine  the  significance  of  our 
effects. 

The  advantages  of  using  the  .Master  Experimental  Design  are 
obvious,  (l)  We  get  a  maximum  amount  of  information  for  our  experi¬ 
mental  dollar;  (2;  wo  know  the  extent  of  confounding  in  our  experiment 
before  the  experiment  is  run;  (3)  if  we  need  to  pick  out  responses 
that  may  be  non-linear,  we  can  always  use  our  fractional  factorial 
as  the  basis  for  further  investigation  using  response  surface  tech¬ 
niques.  The  analysis  of  any  fractional  factorial  experiment  is 
straightforward,  furthermore,  the  analysis  of  a  fractional  factorial 
experiment  will  easily  yield  a  regresaion  equation  relating  the 
change  in  output  to  the  change  in  the  input  variable. 


Suninary 

What  we  have  shown  is  not  impossible,  takes  little  if  any  extra 
effort,  and  is  being  uoed  by  many  of  your  competitors.  Let's  get 
the  most  missile  per  dollar  by  working  smarter,  not  harder. 
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ONE  HUNDRED  PERCENT  PRODUCTION  TESTING 
OF  CRITICAL  COMPONENTS 


ARTHUR  E.  WOOD,  JR. 
LEACH  CORPORATION 
LEACH  RELAY  DIVISION 


THE  REQUIREMENT: 

The  electrical  systems  of  air-borne  vehicles  have  always  required 
components  of  high  performance  and  high  reliability,  capable  of  wlth- 
«t«nd!ng  the  environments  of  flight. 

The  present  day  missile,  which  subjects  Its  components  to  environ¬ 
ments  considered  Impossibly  severe  a  few  years  ago,  and  Its  demand  for 
the  highest  possible  degree  of  reliability  requires  that  steps  be  taken 
to  further  improve  both  performance  and  reliability. 

THE  SOLUTION: 

A  positive  step  toward  these  ends  Is  the  performance  of  environmen¬ 
tal  testing  on  a  production  basis. 

The  fact  that  extremes  of  environment  exist  in  missile  applications 
is  well  known.  Components  may  be  exposed  to  high  or  low  temperatures, 
vibration  shock,  acceleration  and  vacuum  in  addition  to  the  deteriorating 
effects  of  humidity,  dust,  or  corrosive  atmosphere. 

NORMAL  PROCEDURE: 

In  most  instances  in  component  manufacture,  performance  characteristics 
and  environmental  capabilities  are  considered  during  design  stages  and 
are  subject  to  study  and  proof  testing  prior  to  the  start  of  production. 

Initial  production  is  then  subjected  to  a  program  of  formal  quali¬ 
fication  testing  to  the  requirements  of  one  or  more  applicable  perfor¬ 
mance  or  environmental  specifications.  The  results  of  these  tests  are 
compiled  in  the  form  of  certified  reports  which  are  available  for  the 
customers'  use  in  evaluating  capabilities  or  for  obtaining  use  approval 
from  the  mi  I i tary  . 

At  this  point,  normal  quality  control  procedures  are  applied  to  pro¬ 
duction  of  the  device  with  acceptance  and  periodic  destructive  testing 
i  nc I uded  . 

CRITICAL  COMPONENTS : 


Through  envi ronmentai  testing  of  100%  of  relays  destined  for  use  In 
critical  circuits,  it  is  possible  to  provide  a  far  greater  assurance  of 
performance  than  can  be  obtained  through  normal  quality  control  procedures 


and  do  much  to  enhance  the  reliability  of  a  complex  system  exposed  to 
extremes  of  environment. 

THE  TESTS: 


Depending  upon  conditions  of  application,  the  extent  of  the  test 
program  applied  varies  with  different  relays  and  customer  requirements* 

One  production  test  program  includes  the  measurement  of  relay  operate 
values,  dielectric  strength  and  insulation  resistance,  coil  resistance, 
and  contact  resistance  under  toad  while  subject,  in  turn,  to  high  tempera¬ 
ture  (+  85  *C),  low  temperature  (-55  *C),  vibration  (10G  to  2000  CPS), 
and  sustained  acceleration  ( I 5G) • 

Any  relays  exhibiting  variations  In  characteristics  outside  narrow 
limits  are  rejected . 

In  addition,  each  relay  is  required  to  withstand  a  specified  value 
of  shock  without  contact  disturbance  and  to  demonstrate  integrity  of 
hermetic  seal  at  the  conclusion  of  environmental  exposure. 

Another  type  of  testing  involves  energizing  and  de-energizing  the 
relay  for  several  thousand  operations  while  monitoring  contacts  for  pro¬ 
per  functioning  during  each  individual  operation.  Relays  displaying  a 
single  Instance  of  excessive  contact  resistance  are  rejected. 

It  has  been  found  that  the  smell  quantity  of  relays  disposed  toward 
contact  discontinuity  generally  axhlblts  the  tendency  within  the  first 
2,000  operations  and  It  is  possible  to  eliminate  such  units  by  monitored 
cycling.  It  has  also  been  found  that  relays  of  the  "contaminant-free" 
type  (capable  of  meeting  the  "Minimum  Current"  requirements  of  MIL-R-6I06C) 
display  about  1/20  the  miss  rate  of  relays  of  conventional  construction. 

Other  test  prog. ams  Include  portions  or  combinations  of  the  afore¬ 
mentioned  evaluations.  For  example,  vibration  plus  operational  tests  are 
performed  on  e  number  of  relay  types. 

EXPERIENCE: 


In  the  course  of  testing  over  approximately  a  three-year  period,  a 
wide  range  of  failure  rates  have  been  experienced.  Certain  relay  types 
have  displayed  zero  failure  rates. 

In  other  cases,  relays  which  were  tested  close  to  their  ultimate 
capabilities  have  displayed  temporary  rates  In  excess  of  50%  when  some 
characteristic  was  not  adequately  controlled.  Such  an  occurrence  of 
course  signals  the  need  for  immediate  remedial  action. 

In  general,  rejection  rates  of  the  order  of  a  oer  cent  are  experienced. 
(Figure  z) 
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THEORY; 


Production  environmental  testing  is  merely  an  extension  of  normal 
acceptance  testing  procedures  In  which  characteristics  are  measured  under 
ruom  cond I t Ions . 

The  concept  has  basis  In  the  generally  accepted  relationship  of 
failure  rates  of  manufactured  devices  to  time  or  cycles  of  use.  The 
form  of  a  plot  of  this  relationship  is  a  "U"  with  higher  failure  rates 
existing  In  the  Initial  period  of  life,  then  dropping  to  a  lower  constant 
value  until  the  point  of  wear-out  Is  reached,  at  which  time  the  curve  of 
failure  rate  again  rises  sharply.  (Figure  I) 

Production  testing  serves  to  eliminate  those  units  which  would  cause 
the  Initial  levels  of  failure. 

Verification  of  the  shape  of  this  life  curve  has  been  obtained 
through  tasting  of  over  25,000  relays. 

WHAT  THE  PROGRAM  OFFERS: 

In  addition  to  the  obvious  advantage  of  providing  relays  which  have 
already  demonstrated  their  capabilities  in  the  extremes  of  environment 
of  their  Intended  application,  a  program  of  this  type  can  also  offer  other 
contributions  to  overall  component  reliability. 

In  the  normal  production  and  sale  of  relays,  there  is  almost  no 
feedback  of  specific  reliability  (or  unreliability)  information  relating 
to  performance  in  actual  application.  Because  of  this,  evaluation  of 
relay  designs  and  their  extension  by  improvement  and  modification  into 
new  designs  (the  advancement  of  the  state  of  the  art)  must  be  based  upon 
the  testing  performed  on  a  relatively  small  number  of  units  during  develop¬ 
ment  and  qualification.  Because  of  the  small  quantity  of  data  which 
such  testing  generates  and  because  defects  of  low  probability  of  occur¬ 
rence  may  not  even  be  demonstrated,  a  program  of  environmental  testing  of 
production  quantities  of  units  offers  a  source  of  information  on  perfor¬ 
mance  which  can  be  of  great  value  to  the  designer  and,  through  product 
improvement,  to  the  user. 

Failures  which  occur  in  the  course  of  production  environmental  test¬ 
ing  are  analyzed  and  causes  reported  to  the  engineering  department.  In 
the  experience  of  leach,  a  number  of  very  subtle  design  changes  resulting 
in  Improved  product  reliability  have  been  suggested  by  data  accumulated 
during  production  testing. 

In  addition,  data  is  provided  for  control  of  process  as  the  search¬ 
ing  evaluation  of  environmental  testing  gives  Immediate  indication  of  the 
existence  of  defect  producing  situations- 
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If  satisfactory  levels  of  reliability  are  to  be  achieved  In  the 
application  of  any  component,  It  is  absolutely  necessary  that  the  condi¬ 
tion;  of  environment  to  which  the  component  is  to  be  subjected  are  properly 
specif!*'4.  No  amount  of  testing  In  one  level  of  environment  will  insure 
t: 'liability  if  actual  environments  differ  and  are  in  excess  of  component 
rating. 

An  example  of  such  a  situation  is  to  be  found_ln  a  recent  missile 
failure.  A  relay  which  had  been  produced  and  tested  (by  the  user)  to 
mee;  exacting  standards  was  subjected  in  the  application  environment  to 
sho^k  accelerations  of  two  and  one  half  times  the  specified  relay  capa¬ 
bility.  In  this  situation,  contact  disturbance  resulted  in  destruction 
of  ihe  missile.  The  means  of  prevention  of  this  failure  lay  .pot  In  com¬ 
ponent  test  but  in  correct  evaluation  of  environment,  prior  to,  rather 
than  after  the  occurrence. 

NOT  A  LI  HI  TAT  I  ON: 

The  fear  that  production  testing  may  "wear  out"  a  device  Is  not 
valid,  if  designs  are  adequate  to  the  rating  and  if  the  warning  signals 
of  high  failure  rates  are  observed.  Proper  evaluation  of  failure  data 
can,  in  fact,  show  such  things  as  inadequacy  of  design,  premature  wear 
out,  or  lack  of  control  in  process. 

SUMMARY: 

Production  environmental  testing  offers  a  means  of  producing  com¬ 
ponents  of  proven  environmental  capabilities.  In  addition,  the  informa¬ 
tion  collected  in  such  testing  can  be  of  great  value  to  both  designer 
and  those  responsible  for  quality  control.  The  total  result  is  signifi¬ 
cant  increase  in  the  reliability  of  the  device  which  is  so  tested. 


THE  VALUE  OF  P.l.T.  AS 
A  QUALITY  CONTROL  FUNCTION 

ROBERT  L.  ST ALLARD 
STAFF  ENGINEER 
THE  MARTIN  COMPANY 
DENVER,  COLORADO 

Production  Environmental  Testing  le  far  from  a  new  Idea;  for  ax* 
i  ample,  It  wasn't  many  years  ago  that  before  a  customer  would  plunk  out 

cash  for  a  new  flivver  he  would  insist  upon  being  allowed  to  drive  the 
car  over  the  roughest  road  and  up  the  steepest  hill  in  town.  Sometimes 
he  would  insist  upon  taking  the  car  to  a  neighboring  village  whose  favor - 
automobile  test  ground  boasted  a  steeper  hill  or  a  rougher  road. 

This  Production  Environmental  Test  the  dealer  rightfully  fearad,  since 
he  knew  that  if  the  tin  little  rattled  too  much  on  the  washboard  road  or 
refused  to  climb  the  hill  in  high  gear,  the  customer  would  take  his  cash 
elsewhere . 

In  those  golden  days  of  the  Model  T,  many  cars  wouldn't  evan  make 
the  hill  in  low  gear  and  it  was  necessary  to  turn  around  and  back  up  to 
make  the  top.  Also,  whoever  heard  of  a  Model  T  that  didn't  rattle  right 
along.  Thus,  anybody  who  did  not  perform  such  a  Production  Environ- 
mental  Test  was  considered  an  idiot  by  his  friends  and  a  sucker  by  the 
dealer.  However,  nowadays  that  the  Automobile  Designs  and  Production 
Techniques  have  been  almost  completely  "debugged",  the  need  to  see  if  an 
automobile  can  charge  up  a  hill  prior  to  its  procurement  is  unnecessary. 

GENERAL  DESCRIPTION  OF  THE  P.E.T,  PROGRAM 

In  the  field  of  Missile  Technology  pertaining  to  component  design 
and  fabrication,  sufficient  fully  developed  and  debugged  components  to 
completely  outfit  a  missile  are  not  available  "off  the  shelf".  For  this 
reason,  a  Production  Environmental  Test  Program  was  instituted  by  The 
Martin  Company. 

The  Martin  Company's  Production  Environmental  Teat  (P.E.T.)  Pro* 
gram,  which  is  operated  by  an  Engineering  Group  within  Quality  Control, 

*  consists  of  a  1007.  inspection,  Including  functional  and  environmental 

te6ts,  of  the  total  production  of  s  selected  group  of  components.  The 
primary  purpose  of  The  Martin  Company's  P.E.T.  Program  is  to  monitor 
Quality  Control  and  Manufacturing  Control  of  the  component  manufacturer. 

Significantly,  In  running  the  P.E.T.  Program,  it  has  been  found 
that  de b1 gn  deficiencies  become  self  evident.  Informstion  concerning 
thece  deficiencies  may  be  sent  to  the  Vendor  lniDedlately  and,  therefore. 
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design  Improvement  does  not  need  swalt  the  results  of  Field  Tests.  This 
ability  of  P.E.T.  to  detect  design  deficiencies,  although  It  has  not 
been  considered  the  basic  purpose  of  the  Program,  has  wrought  many  useful 
design  changes. 

The  components  considered  for  Inclusion  in  the  P.E.T.  Program  were 
those  which  are  considered  to  be  of  prime  Importance  in  terms  of  the 
success  of  the  mission  of  the  missile. 

The  criteria  used  for  this  selection  of  items  and  tests  to  be  per¬ 
formed  is: 

1.  Items  selected  must  have  potential  modes  of  failure  which  could 
result  in  a  serious  loss  of  objective  in  the  Flight  Test  Pro- 
gram. 

2.  There  must  be  insufficient  confidence  that  normal  techniques 
such  as  in  line  Inspection  and  bench  testing  can  eliminate 
failures  In  these  modes. 

3.  The  P.E.T.  test  for  the  unit  must  significantly  Increase  the 
probability  of  detection  of  deficiencies  in  the  unit. 

4.  Previous  test  history  on  any  item  does  not  already  give  suffl- 
cent  confidence  that  the  Item  is  adequately  developed  and  the 
Production  processes  are  under  good  control. 

In  the  application  of  the  above  ground  rules  for  the  selection  of 
the  components  to  be  Included  in  the  P.E.T.  Program,  Engineering  Judg¬ 
ment  was  exercised.  For  example,  it  is  difficult  to  say  exactly  which 
component  can  or  cannot  by  itself  cause  a  catastrophic  missile  failure, 
If  it  falls  to  operate  properly.  This  is  particularly  true  in  the  gray 
area  wherein  a  component  operates  out  of  specification  by  only  a  slight 
margin.  The  results  of  the  selection,  made  according  to  these  ground 
rules,  is  shown  in  Table  I. 

TABU?  I 

List  of  Components  Presently  Included  In  the  P.E.T.  Program 


Quantity 


Position  Gyro  -  Programmer  Package 
3-Axls  Rate  Gyro  Package 
Magnetic  Amplifier  Autopilot  Package 
Servo  Magnetic  Amplifier  Package 
Zener  Diode  Program  Timer  Package 
Hydraulic  Servo  Actuators 
Liquid  Level  Sensor 
Relay  Panel  Assembllea 
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10  .Telemetering  Signal  Conditioners 

1  Telemetering  Package 

1  Command  Control  Receiver 

3  2nd  Instrument  Power  Supplies 

2  Inverters  (28  VDC  to  115  V  400n3P) 

1  Plunger  Switch 

2  Relays 

1  Hi -Press  Relief  Valve 

1  3 -Way  Electrically  Operated  Pneumatic  Valve 

1  Power  Supply 

1  Rotary  Frequency  Converter 

1  Hydraulic  Regulating  Unit 

4  Tank  Vent  &  Relief  Valves 

6  Tank  Pressure  Regulators 

7  Pneumatic  &  Hydraulic  Quick  Dlsconnecta 

4  Fuel  &  Lox  Shut-Off  Valves 

1  Trim  Potentiometer  Package 

68  Total  number  of  components  in  the  present  P.I.T. 

Program. 

Percentage  of  Components  In  P.E.T.  Program  by  General  Categories 
37%  Mechanical 
28%  Electro-Mechanical 
35%  Electrical/Electronic 

The  type  of  components  Included  in  the  three  general  categories 
listed  above  are  as  follows: 

Mechanical 

Valve6,  Regulntors,  Check  Valves,  Qvick  Disconnects 
Electro -Mechanics'. 

Hydraulic  Actuators,  Gyros,  Relayo,  Switches,  Inverters, 
Solenoid  Valves 

Electrlcal/Electronlc 

Autopilots,  telemetering,  power  supplies 


The  only  environment  to  which  the  selected  component#  are  subjected 
In  the  P.E.T.  Program  at  present  Is  vibration,  with  the  exception  that 


cryogenic  components  ere  subject  to  cryogenic  temperatures  and  vibra¬ 
tion  simultaneously.  During  the  vibration  testing,  all  components  are 
functionally  operated  In  Che  same  manner  that  they  are  operated  during 
missile  flight. 

The  level  of  vibration  used  In  the  P.E.T.  Program  vaa  obtained  from 
basic  contractual  environmental  test  specifications.  The  levels  of  the 
vibration  tests  run  In  the  P.E.T.  Program  vary  from  about  20X  to  100%  of 
the  specification  magnitude.  In  no  cases  were  the  specification  magni¬ 
tudes  exceeded.  -Time  durations  of  all  tests  vere-greatly  reduced  from 
specification  requirements  and  all  resonant  search  and  dwell  tests  con¬ 
tained  In  the  basic  specifications  were  eliminated. 

In  the  present  P.E.T.  Program  the  percentage  of  components  being 
tested  at  various  percentages  of  design  levels  is'  shown  in  Table  II. 

TABLE  II 

7 OX  of  the  components  are  tested  at  100X  of  design  level. 

7X  of  the  components  are  tested  between  SOX  and  100X  of  design 
level. 

23X  of  the  components  are  tested  below  50X  of  design  level. 

At  the  present  time,  consideration  la  being  given  to  expanding  the 
P.E.T.  Program  to  Include  other  environments  such  as  temperature,  alti¬ 
tude,  etc.  In  examining  the  deleterious  effect  of  these  other  environ¬ 
ments  upon  components,  consideration  will  also  be  given  as  to  the  syner¬ 
getic  effect  of  these  environments.  It  Is  not  expected  that  combination 
testing  or  the  Inclusion  of  other  environments  will  be  employed  as  part 
of  P.E.T.  In  the  near  future,  or  at  least  not  until  Initial  Investiga¬ 
tive  tests  can  be  evaluated. 

OTHER  ENVIRONMENTAL  TESTING  AS  RELATED  TO  P.E.T. 

Prior  to  acceptance  of  a  particular  component  for  inclusion  in  the 
R  and  D  configuration  of  a  f  ight  vehicle,  it  is  subjected  to  Evaluation 
Tests  by  Design  Engineering.  The  purpose  of  these  tests  is  to  provide 
limited  qualification  for  the  component  so  the  program  may  be  activated 
In  a  minimum  time  span.  Hence,  only  those  environments  that  would  have 
an  immediate  effect  upon  the  successful  flight  of  an  R  &  D  flight  vehi¬ 
cle,  such  as  vibration,  high  and  low  temperature,  etc.,  are  included. 
However,  all  testing  that  is  done  in  the  Engineering  Component  Evaluat¬ 
ion  Program  is  performed  at  the  levels  specified  in  Environmental  Test 
Specification  provided  by  the  customer. 

Prior  to  the  fabrication  of  Operational  Flight  Vehiclea,  It  is  re¬ 
quired  that  a  full  scale  Qualification  Program  is  accomplished.  In  this 


program,  che  component  will  be  subjected  to  *11  applicable  environ¬ 
ments  a*  described  in  the  contractual  environmental  specification. 

The  time  lag  between  the  completion  of  the  Evaluation  Testing  and 
Qualification  Testing  is  advantageous.  The  necessity  of  changing  the 
component  characteristics,  design,  or  manufacturing  techniques  as  un¬ 
covered  by  P.E.T.  and  R  &  D  missile  operations  can  be  made  prior  to  the 
comaencement  of  an  expensive  Qualification  Program.  Often  there  is  even 
sufficient  time  for  complete  evaluation  of  the  "fix"  to  determine  wheth¬ 
er  it  has  accomplished  its  purpose. 

THE  PHILOSOPHY  OF  P.E.T. 


In  the  production  of  large  complex  devices  such  as  missiles,  which 
consists  of  many  components,  it  would  be  ideal  if  the  development  of 
these  components  could  be  pursued  in  a  logical  step-by-step  manner.  A 
generalized  description  of  such  a  step-by-step  method  is  as  follows! 

1.  Definition  of  the  requirements. 

2.  Production  and  test  of  breadboard  models. 

3.  Production  of  a  prototype,  which  is  subjected  to  environ¬ 
mental  tests. 

4.  Re-design  of  the  component,  as  dictated  by  the  results  of  the 
prototype  tests. 

5.  Production  of  several  prototypes  with  production  tooling  and 
production  techniques. 

6.  Full  scale  qualification  testing  of  all  these  prototypes  pro¬ 
duced  by  production  methods. 

7.  Re-design  and  retest  of  the  production  prototypes,  as 
necessary. 

8.  Final  production. 

If  all  the  components  Included  in  a  missile  design  had  gone  thro¬ 
ugh  this  process  of  evolution  before  installation  In  a  missile,  missile 
failures  would  be  few  and  far  between.  However,  we  cannot  afford  such 
a  luxury  due  to  the  compressed  production  schedules  dictated  by  the 
Department  of  Defense,  nor  do  ve  advocate  the  use  of  such  a  procedure. 
For,  although  admittedly  it  is  ideal,  we  would  never  get  the  work  done. 

Therefore,  for  the  moBt  part,  missile  R  &  D  production  is  pursued 
with  components  whose  selections  were  based  upon  Engineering  evaluation 
of  preproduccion  prototypes.  In  taking  this  expeditious  approach,  some 
things  are  bound  to  "slip  through  the  crack".  It  is  the  purpose  of 
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P.E.T.  to  detect  workmanship  and  manufacturing  deficiencies  which  can¬ 
not  be  detected  in  a  program  operated  on  a  compressed  time  scale  where¬ 
in  many  steps  of  idealised  component  production  must  be  eliminated. 
Specifically,  the  manners  in  which  a  F.K.T.  Program  can  aid  production 
are  tabulated  below: 

1.  By  constant  testing  in  a  F.K.T.  Program,  deficiencies  in 
Manufacturing  procedures,  piece  part  selection,  assembly 
procedures,  and  Quality  Control  nay  be  detected  and  eradicated. 

2.  Production  may  be  monitored  for  any  sudden  increase  in  fail¬ 
ure  rate,  which  can  be  caused  by  carelessness,  or  loss  of 
manufacturing  control,  or  Quality  Control,  in  the  middle  of 
production.  With  a  P.E.T.  Program,  such  a  condition  will  be 
Immediately  self-evident  and  appropriate  action  may  be  taken 
immediately.  Without  a  P.E.T.  Program,  such  a  condition  might 
not  be  detected  until  field  tests  four  or  five  months  later. 

The  partial  elimination  of  resulting  problems  of  retrofitting 
components  in  widely  scattered  missiles,  make  this  perhaps 
one  of  the  most  advantageous  aspects  of  the  P.E.T.  Program. 

3.  A  P.E.T.  Program  will  keep  vendors  on  their  toes.  It  functions 
as  a  relentless  check  of  the  vendors  capability  to  produce 
quality  components. 

4.  A  F.K.T.  Program  will  act  as  a  Quality  Control  screen  of  com¬ 
ponent  production,  for  occasional  and  Inadvertent  errors  by 
the  manufacturer.  This  property  of  P.E.T.  is  only  valid,  how¬ 
ever,  after  the  component  exhibits  a  reasonable  failure  rate. 

A  P.E.T.  Program  will  produce  many  valuable  secondary  effects,  two 
of  which  are  tabulated  below: 

1.  A  P.E.T.  Program  can  provide  a  means  through  constant  testing, 
of  "de-bugglng"  troubles  in  designs. 

2.  In  the  initial  stages  of  the  program,  the  components  whose  de¬ 
sign  is  hopeless  can  be  eliminated.  These  are  designs  which 
somehow  passed  the  original  evaluation  tests,  but  somehow  the 
vendor  Just  cannot  produce  workable  components  in  quantity. 

In  pursuing  a  program  in  this  manner,  it  should  be  expected  that 
Initially  fairly  high  failure  rates  would  exist  and  that  a  few  designs 
would  have  to  be  eliminated.  This  does  not  mean  that  the  original  Engin¬ 
eering  Judgment,  as  expressed  in  the  original  selection  of  the  original 
group  of  components,  was  poor.  In  fact,  we  have  found  that  in  the  maj¬ 
ority  of  cases,  this  Engineering  Judgment  was  excellent.  The  main  rea¬ 
son  of  these  Initial  failures  Is  that  the  design  is  new  and  the  vendor 
has  had  little  experience  In  producing  and  testing  this  particular  com¬ 
ponent  . 
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The  Initial  results  obtained  in  terms  of  component  failures  for 
the  first  three  months  of  the  operation  of  the  P.E.T.  Program  are  shown 
in  Figure  1.  This  rather  high  Initial  rate,  as  previously  stated,  was 
to  be  expected.  However,  as  usual,  a  hue  and  cry  was  raised  that  the 
test  was  at  fault.  In  order  to  answer  this  question,  as  to  test  validi¬ 
ty  of  testing  done  in  the  P.E.T.  Program,  a  thorough  Investigation  was 
made  of  all  test  laboratories. 

IMPROVEMENT  IN  TESTING  TECHNIQUES 

The  investigation  of  the  test  laboratories  brought  about  by  the  high 
initial  failure  rate  is  a  very  Intersting  sidelight  of  the  P.E.T.  Pro¬ 
gram.  Although  this  investigation  brought  to  light  some  rather  deplor¬ 
able  conditions,  it  did  not  invalidate  the  testing  which  had  been  accom¬ 
plished.  It  was  found  that  a  very  large  percentage  of  the  test  labora¬ 
tories  had  little  or  no  control  over  contamination  and  showed  a  lack  of 
understanding  of  the  basic  fundamentals  of  vibration  testing.  In  short, 
good  testing  practices  were  being  flagrantly  violated. 

To  alleviate  this  condition,  a  system  of  laboratory  evaluation  was 
initiated  and  the  P.E.T.  requirements  were  "tightened  up",  particularly 
in  the  area  dealing  with  vibration  fixtures,  contamination,  and  instru¬ 
mentation.  Laboratories  that  were  found  unacceptable  were  informed  of 
their  deficiencies  and  Instructed  as  to  the  corrective  action  necessary. 
This  vendor  educational  program  for  which  P.E.T.  was  primarily  responsi¬ 
ble,  although  it  does  not  have  any  direct  bearing  upon  the  improvement 
of  the  components,  is  one  of  the  distinct  advantages  that  were  obtained 
from  the  P.E.T.  Program.  Testing  done  today  is  repeatable,  dependable, 
and  provides  more  information  than  was  available  in  the  initial  stages 
of  the  Program. 


POOR  DESIGNS  "WEEDED  OUT"' 


Examination  of  Figure  I  indicates  that  there  was  s  fair  percent¬ 
age  of  components  that  had  failure  rates  above  60X.  The  items  in  this 
category  were  examined  critically  for  the  cause  of  the  high  failure 
rate.  Included  in  this  group,  were  four  items  Chat  were  cancelled  due 
to  Inadequate  design,  during  the  first  four  months  of  the  program.  For 
two  of  these  items,  five  each  were  built,  and  none  could  pass  P.E.T. 
even  after  repeated  reworks;  for  the  third  item,  51  were  built  and  r  ily 
4  were  able  to  pas6  P.E.T.;  for  the  fourth  item,  82  were  built  and  o.iy 
25  were  barely  able  to  pass  P.E.T.  after  several  reworks  and  retests. 
The  failures  of  these  items  occurred  during  the  vibration  portidn  of 
the  P.E.T.  Program;  not  during  the  pre-test  functional  and  hence  would 
not  have  been  detected  without  P.E.T.  Units  that  passed  P.E.T.  in  the 
last  category  were  installed  in  flight  vehicles  but  were  not  function- 
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ally  connected  to  the  missile  system.  Telemetering  records  indicated 
that  the  four  flights  In  which  they  were  installed,  the  missile  would 
have  been  destroyed  had  they  been  functionally  active.  The  other  three 
items  were  of  such  a  nature  that  they  could  not  have  been  installed  in 
the  missile  system  and  yet  remain  inactive. 

The  components  are  a  good  example  of  the  condition  under  which  F.B.T. 
cannot  be  used  as  a  Quality  Control  screening  process.  The  failure  rate 
of  these  components  was  so  great  in  P.E.T.  that  no  faith  was  placed  in 
thobe  few  units  which  were  able  to  pass  a  F.B.T.  test.  The  flight  test 
results  proved  this  decision  correct. 

CENTRAL  IMPROVEMENT  OF  COMPONENT _ QUALITY 

A  general  improvement  of  component  quality  is  illustrated  in  Fig¬ 
ures  1  and  2.  Through  the  end  of  December  1959  In  the  F.B.T.  Program, 
lCd43  tests  have  been  conducted  with  a  total  of  2,133  failures,  or  an 
average  failure  rate  of  19.51.  It  is  not  expected  that  this  failure 
rate  will  significantly  decrease  throughout  the  F.B.T.  Program.  This 
is  due  to  the  fact  that  when  the  production  of  the  component  has  been 
proven  under  sufficient  control,  it  will  be  deleted  from  the  F.C.T. 
program.  Hence,  the  P.E.T.  Program  will  include^ only  components  whose 
failure  rate  is  sign.4. f leant .  To  date,  20  components  have  been  deleted 
from  the  P.E.T.  Program  b^auae  of  this  reason. 

In  identifying  the  Improvement  in  component  quality,  it  la  first 
necessary  f:o  categorise  failures  into  types.  Table  III  lists  the  fail¬ 
ure  types  ns  assigned  to  various  categories. 

TABLE  HI 

Type  Pal lures  as  Assigned  to  Various  Categories 

1.  Manufacturing/Qunlity  Control 

a.  Improper  Assembly 

b.  Poor  Finishes 

c.  Poor  Materials 

d.  Incorrect  Dimensions 

e.  Improper  Calibration  Procedures 

f.  General  contamination  in  components 

g.  Burrs  not  removed,  etc. 

2.  Design 

a.  Design  Includes  only  items  for  which  physical  changes  in 
design  were  necessary  to  alleviate  the  problem. 
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3.  Teat  Error 


a.  Test  Error  is  associated  vlth  failures  in  which  the  appli¬ 
cation  of  too  much  environment  caused  a  failure . 

4 .  Unknown 

a.  This  is  applicable  to  random  failures  in  which  failure 
analysis  indicated  possible  causes  of  the  trouble  but  no 
definite  correlation  could  be  made.  Also,  those  failures 
for  which  no  explanation  could  be  made. 

MOTE:  For  all  the  failures  Included  in  this  report,  failure 
analysis  has  been  conducted. 

It  should  be  stated  that  it  is  difficult  to  segregate  all  failures 
into  the  various  categories  as  listed.  For  example,  an  excellent  design 
can  stand  more  Manufacturing  and  Quality  Control  flaws  than  can  be  a 
marginal  design.  However,  it  is  felt  that  the  failure  causes  as  pre¬ 
sented  are  reasonably  accurate. 

The  breakdown  of  the  total  failures  to  date  are  presented  in  Table 


TABLE  17 

Tabulation  of  Cause  of  Failures  for  all  Failures  to  Date 


Cause  of  Failure 


Percent 


Design 

Manufacturing/Quality  Control 


47. 9% 


41.9% 


Test  Error 


Unknown 


Figure  1  is  a  histogram  of  the  history  of  the  P.E.T.  Program.  The 
percentage  of  the  component  types  that  exhibit  various  failure  rates  are 
indicated.  Examination  of  the  figure  shows  that  at  the  start  of  the  Pro¬ 
gram,  only  24%  of  the  various  components  being  tested  had  a  failure  rate 
of  less  than  5%,  whereas  now  57%  of  the  components  being  tested  have  a 
failure  rate  of  less  than  5%.  This  percentage. is  actually  much  greater 
due  to  the  20  components  that  have  been  dropped  from  the  P.E.T.  Program 
since  they  continuously  exhibited  failure  rates  in  this  range.  As  a 
basis  of  comparison,  the  average  failure  rate  is  also  indicated. 

A  trend  Is  Indicated  that  components  are  real  good  or  real  bad  as 
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shown  in  figures  for  November  and  December;  wherein  the  failure  rate  in 
the  5  to  10X  range  la  significantly  lower  than  the  0  to  S%  range  and  the 
higher  failure  percentiles.  In  addition,  examination  of  the  failure  rate 
of  individual  components  Indicates  that  when  a  component  has  been  im¬ 
proved  to  the  point  its  failure  rate  is  below  5X,  this  failure  rate  will 
be  maintained  throughout  that  production  run. 

Figure  2  shows  the  failure  rate  plotted  versus  the  length  of  time 
the  P.E.T.  Program  has  been  operating.  The  Increase  in  failure  rate 
during  April  and  Hay  of  1959  coincides  with  the  introduction  of  new 
missile  lots  which  resulted  in  changes  in  component  operating  require¬ 
ments  and  the  addition  of  some  new  components.  Thus,  this  Increase  in 
failure  rate  should  be  expected. 

The  final  average  failure  rate  reached  at  the  end  of  1959  of  about 
15X  is  actually  lower  due  to  the  20  components  dropped  from  the  P.E.T. 
Program  which  exhibited  satisfactory  performance. 

Figure  3  illustrates  in  what  portion  of  a  Production  Environmental 
Test  components  failed;  l.e.,  1.  During  the  pre-functional  test,  2. 
During  the  Environmental  portion  of  the  test,  or  3.  During  the  post- 
environmental  functional  test.  This  data  is  presented  for  all  the 
components  collectively. 

The  cross-hatched  area,  which  Includes  the  Environmental  Test  and 
the  Post-Environmental  Functional  Test  portions  of  the  P.E.T.  Program, 
represents  the  failures  of  components  that  probably  would  not  have  been 
detected  without  a  P.E.T.  test.  This  amounts  to  about  751  of  the  fail¬ 
ures  that  occurred  throughout  the  P.E.T.  Program. 

A  large  portion  of  the  failures  that  occurred  during  the  vibration 
portion  of  P.E.T.  were  of  the  Intermittent  type.  That  is,  when  the  com¬ 
ponent  was  functioned  under  vibration,  it  failed  to  function  within 
specification  limits.  Yet,  when  the  vibration  was  removed,  the  compon¬ 
ent  functioned  satisfactorily.  Relays  that  chattered  under  vibration 
are  a  good  example  of  this  condition. 

PROBLEMS  IDENTIFIED  AND  SOLVED  DURING  PRODUCTION  ENVIRONMENTAL  TEST 


Listed  in  Table  V  are  the  number  of  components  and  the  number  of 
failures  for  these  components  for  which  the  cause  of  failure  has  been 
identified  and  a  proven  solution  identified.  This  action  is  a  direct 
result  of  the  Production  Environmental  Test  Program. 


TABLE  V 


fc 

f. 

V 

s 

I 


£ 


L* 

r- 

r. 


Number  of  Problema  In  Component  Design  or  Fabrication  Procedures  Dis¬ 
covered  In  the  Production  Environmental  Teat  Program  for  which  Proven 
Solutions  have~~^een  Pound;  — 


Number  of  Component  Types  Problems  Identified 


Manufacturlng/Quallty  Design 

Control  Problems  Problems 


Mechanical  Components  15  14 

19  different  components 
represented 

Electro-Mechanical  Components  11  10 

13  different  components 
represented 

Electrical/Electronic  Components  16  19 

12  different  components 
represented 


Total  number  of  components  for  which  failure  causes  have  been  Identified 
and  corrected— 44 . 

Total  number  of  failure  problems  Identified  and  corrected— 85 ♦ 

NOTE:  When  one  vendor  manufacturers  more  than  one  component,  the 
same  Manufacturing  and  Quality  Control  problems  usually 
occurred  on  all  components. 

QUALITY  IMPROVEMENT  IN  CROUPS  OF  LIKE  COMPONENTS 

Figures  4  through  6  describe  the  failure  history  In  the  P.E.T.  Pro¬ 
gram  for  several  Gets  of  like  components  manufactured  by  different 
vendors . 

Figure  4  represents  the  failure  rate  of  similar  mechanical  compon¬ 
ents  manufactured  by  a  single  vendor.  This  vendor  had  serious  trouble 
with  the  designs  and  manufacturing  procedures  initially.  These  diffi¬ 
culties  were  great  enough  so  that  In  the  initial  stages  of  the  program, 
serious  consideration  was  given  to  dropping  this  vendor  from  the  program. 

The  failure  rate  of  these  unitB  demonstrated  to  the  vendor  the  need 
to  completely  revamp  his  manufacturing  procedures  and  design.  In  making 
these  changes,  which  were  numerous,  the  vendor  was  able  to  use  the  P.E.T. 
Program  to  monitor  the  degree  of  success  of  the  changes  made  while  simul¬ 
taneously  allowing  production  to  continue. 
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In  Addition  to  changing  manufacturing  procedures,  the  vendor  made 
many  changes  to  his  physical  plant.  Probably  the  most  effective  in  de¬ 
creasing  failures  due  to  manufacturing  causes  was  the  Improvement  in 
Facility  in  the  Dust  Free  Assembly  areas. 

With  this  group  of  components,  the  P.E.T.  Program  produced  per¬ 
haps  the  most  spectacular  results,  particularly  in  view  of  the  fact 
P.E.T.  was  conducted  by  the  vendor  in  his  own  plant.  The  changes  made 
lu  design,  manufacturing  procedures,  and  the  physical  plant  were  of  his 
twn  volition. 

The  results  of  this  action  have  been  rewarding.  The  failure  rate 
has  decreased  from  a  more  or  less  constant  60%  at  the  start  of  the  pro¬ 
gram  to  the  present  20%. 

Figure  S  illustrates  a  group  of  fairly  complex  electro-mechanical 
components  with  which  considerable  design  difficulty  has  been  experienced 
throughout  the  program  up  to  October.  From  then  on  no  more  failures 
have  been  experienced  due  to  design.  Numerous  design  changes  were  made 
and  the  rather  violent  changes  in  the  failure  rate  curve  indicates  their 
success  or  la~k  thereof. 

The  success  of  the  Program  is  evidenced  by  the  failure  rate  in 
December  wherein  65  components  were  tested  with  only  two  failures. 

Figure  6  illustrates  the  failure  history  of  a  group  of  complex 
mechanical  components  for  which  the  P.E.T.  Program  has  not  caused  any 
significant  change  in  failure  rate. 

Throughout  the  P.E.T.  Program  for  these  components,  there  have  been 
Innumerable  changes  made  In  Manufacturing  Procedures,  Quality  Control 
techniques,  and  design.  However,  it  seems  as  fast  as  one  problem  is 
solved,  a  new  and  different  problem  appears  to  take  its  place. 

Although  no  overall  general  improvement  has  occurred  in  this  com¬ 
ponent  throughout  the  P.E.T.  Program,  at  least  P.E.T.  has  shown  and 
continues  to  show  where  the  difficulties  with  these  components  exist. 

QUALITY  IMPROVEMENT  OF  INDIVIDUAL  COMPONENTS 

The  effect  of  P.E.T.  on  Individual  components  is  illustrated  in 
figures  7  and  8.  These  figures  show  the  failure  rate,  and  cause  of  fail¬ 
ure  plotted  vs  Time  for  two  different  components.  These  components  were 
selected  to  show  two  different  typical  situations;, 

1.  A  component  which  had  fair  design  but  control  of  manufacturing 
and  Quality  Control  was  lost  long  after  production  was  started. 
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2.  A  component  with  which  a  greet  deal  of  deelgn  troubles  were 
encountered . 

Figure  7  shows  a  typical  component  with  which  considerable  diffi¬ 
culty  has  been  experienced  throughout  the  P.E.T.  Program.  Initially, 
failures  were  due  to  design.  This  is  to  be  expected  with  any  new  de¬ 
sign.  The  design  was  reworked  and  only  a  few  failures  occurred  after¬ 
wards  which  could  be  attributed  to  dealgn  deficiency.  However,  the 
next  group  of  failures  occurred  with  seemingly  suddenness,  l.e.,  no 
failures  which  were  attributed  to  Improper  assembly  techniques  occurred 
uncll  the  material  problem  was  resolved  and  no  Manufacturing/Quality 
Control  problems  appeared  until  after  the  assembly  problem  was  resolved. 

Admittedly,  it  is  possible  that  the  initial  failures  masked  the 
succeeding  failures.  However,  this  seems  unlikely  due  to  the  nature  of 
the  various  categories  of  failures.  What  seems  most  likely  is  that  this 
vendor  experienced  a  complete  loss  of  Quality  Control  after  eight  months 
of  production. 

THE  ABILITY  TO  DETECT  A  SUDDEN  PE  CREASE  IN  COMPONENT  QUALITY  IN  THE 
MIDDLE  OF  A  PRODUCTION  RUN  13  PERHAPS  OWE  OF  THE  BIGGE8T  ADVANTAGES 

OP  P.E.T. 

Referring  to  the  figures  for  December  1959  in  which  the  greatest 
number  of  components  per  month  were  tested  and  the  lowest  failure  rate 
experienced  with  this  component  occurred,  It  would  seem  that  the  manu¬ 
facturing  deficiencies  with  this  component  were  slowly  being  resolved. 

Figure  8  illustrates  how  P.E.T.  aided  the  "DEBUGGING"  of  a  design 
and  simultaneously  allow  production  to  proceed.  For  this  component,  the 
design  apparently  is  "out  of  the  woods"  as  evidenced  by  the  complete  lack 
of  design  failures  in  November  and  December,  and  for  December  only  one 
failure  occurred  out  of  39  tests  conducted  which  was  due  to  test  error. 

If  this  low  failure  rate  continues,  this  component  soon  will  be  dropped 
from  the  P.E.T.  ProgTam.  The  dip  in  the  failure  rate  curve  in  March 
cannot  be  explained. 

DOES  P.E.T.  WEAR  A  COMPONENT  OUT? 

The  argument  most  -ften  lashed  against  P.E.T.  is,  "You  are  wearing 
the  component  out  with  P.E.T."  "How  do  you  expect  a  missile  to  fly  with 
worn  out  components?" 

Admittedly,  this  argument  has  some  merit.  Most  certainly  it  Is 
highly  undesirable  to  fly  a  missile  with  worn  out  parts.  Yet,  simultan¬ 
eously,  It  is  foolhardy  to  assemble  a  missile  with  inferior  components. 

It  would  be  foolish  to  say  that  P.E.T.  does  not  contribute  to  the  "wear¬ 
ing  out"  process.  However,  when  P.E.T.  requirements  were  determined, 
consideration  was  given  to  prescribing  a  level  that  would  not  seriously 
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contribute  to  “wear  out",  and  yet  provide  a  useful  tool  for  Identification 
of  problems  and  segregation  of  the  usable  from  the  non-usable  components. 


In  vibration  testing,  it  is  difficult  to  specify  what  constitutes 
the  complete  wear  out  of  the  components.  Only  in  cases  wherein  struct¬ 
ural  failure  occurrs  does  an  exact  definition  of  "wear  out"  exist.  .In 
the  case  of  non-destructive  testing,  the  only  way  a  comparison  of  the 
"wear  out"  caused  by  F.S.T.  can  be  made  to  complete  design  life  is  if 
we  consider  a  complete  specification  test  as  completely  exhausting  the 
design  life  of  the  component. 

A  reasonable  way  to  make  such  a  comparison  is  to  compare  the  number 
of  stress  cycles  used  in  a  qualification  test  versus  those  in  a  P.E.T. 
test.  The  number  of  stress  cycles  is  a  function  of  the  product  of  the 
applied  acceleration  and  the  time  for  which  this  acceleration  is  applied. 
The  units  of  this  comparison  function  are  in  "g"  minutes. 

The  stress  level  imposed  upon  a  component  by  a  vibration  test  is  a 
direct  function  of  the  "g"  applied.  Therefore,  since  the  applied  "g" 
in  testing  conducted  in  the  P.E.T.  Program  never  exceeds  specification 
requirements,  a  comparison  of  the  "g"  minutes  in  both  teats  gives  a  good 
Indication  as  to  the  degree  of  "wear  out"  in  P.E.T. 

Such  a  comparison  has  been  made  for  all  testing  done  in  the  P.E.T. 
Program.  Results  of  this  comparison  show  that  the  maximum  amount  of 
design  life  consumed  by  P.E.T.  for  any  component  is  4%. 

Admittedly  this  method  of  comparison  is  not  precise,  but  since  the 
numbers  do  not  turn  out  very  low,  it  does  give  a  strong  indication  that 
we  should  not  expect  "wear  out"  problems.  Results  have  proven  thlc  true. 
We  have  only  observed  one  type  of  component  that  exhibited  "wear  out" 
problems  shortly  after  P.E.T.  was  completed.  This  component  had  a  sub¬ 
standard  bearing  design  which  needed  to  be  changed.- 

SUMMARY 


A  Production  Environmental  Test  Program  is  very  expensive.  On  an 
average,  it  has  been  found  that  the  Increase  In  cost  of  the  individual 
component  ranges  from  10%  to  13%.  We  feel  that  this  Increased  cost  is 
worthwhile.  P.E.T.  has  demonstrated  itself  as  a  necessary  augmentation 
to  the  R  &  D  and  early  production  phases  of  missile  fabrication.  Through 
the  continued  testing  in  P.E.T.,  we  have  learned  a  great  deal  about  the 
components  and  their  operating  characteristics  in  a  relatively  short  time. 

We  have  been  able  to  identify  many  modes  of  failure  by  this  P.E.T. 
Program,  and  by  establishing  corrective  action  to  process  controls  or  by 
making  design  changes  we  have  been  able  to  eliminate  these  problems  at  an 
early  time  In  the  development  cycle. 
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Fig.  3  Distribution  of  Failures  During  PET 
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Fig.  8  Electro-Mechanical  Component  With 
Which  Considerable  Design  Trouble 
Was  Encountered 
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TEST  ERROR 


THE  FALLACY  OF  PET  AS  A  QUALITY  CONTROL  TECHNIQUE 
BY  t  G«org«  A.  Henderson  -  A ray  Rocket  and  Guided  Missile  Agency 


Production  Environmental  Testing  (PET)  is  used  widely  by  the 
electronics  industry  as  a  quality  control  technique  in  an  effort  to 
"debug",  or  to  screen  bad  parts  and  workmanship  from  good.  The 
belief  that  the  application  of  PET  can  increase  the  chance  of  success 
in  missilery  is  a  fallacy  which  will  be  demonstrated  in  this  paper. 

There  seems  to  be  considerable  misunderstanding  as  to  what  is 
meant  by  PET.  PET  Is  the  subjecting  of  100%  of  manufactured  hardware 
to  some  environmental  test,  such  as  ribration,  shock,  acceleration, 
temperature,  etc.,  prior  to  flight  testing  or  service  use.  It 
includes  both  RAD  and  production  lot  equipment  at  component  or 
assembly  level. 

PET  is  perhaps  the  largest  remaining .controversy  in  the  guided 
missile  field,  constituting  expensive  bottlenecks  in  some  manu¬ 
facturers  plants.  It  Is  used  in  an  attempt  to  assure  quality  by 
application  of  what  proponents  may  feel  is  a  panacea,  rather  than  by 
application  of  proper  process  controls  and  suitable  Inspection. 

It  is  the  policy  of  the  Army  Ordnance  Missile  Coranand  that  no 
hardware  which  has  been  subjected  to  a  vibration  test  will  be  used 
in  any  flight  test,  or  otherwise  delivered  to  the  using  troops. 
However,  it  is  required  that  adequate  quantities  of  samples  be  tested 
to  failure  (functional  failure,  not  necessarily  destruction)  in  each 
critical  single  and  combined  environment  (within  the  state  of  the 
art).  This  includes  both  RAD  and  production  lot  equipment. 

Amy  has  had  demonstrated  seme  satisfactory  levels  of  systems 
reliability  without  resorting  to  PET,  simply  by  the  application 
of  sound  development  practices  and  time  tested  quality  control 
techniques. 

One  is  reminded  of  the  story  of  the  easterner  who  drove  his 
automobile  to  the  wett  coast  and  back  again  without  use  of  credit 
cards  or  travellers  cheques  and  without  once  having  to  identify 
himself  or  to  sign  his  name.  He  used  only  time-teoted  cash.  It 
appears  necessary  that  testing  to  failure  of  adequate  quantities 
of  samples  of  hardware  at  all  levels  of  assembly  be  "rediscovered" 
in  order  to  achieve  the  necessary  reliability  in  guided  missile 
systems,  rather  than  continue  to  depend  on  PhT,  which  is  at  best 
no  more  than  a  "crutch". 


The  belief  that  PET  can  serve  as  a  valid  and  useful  means  lor 
detecting  and  correcting  deficiencies,  or  for  debugging,  is  based  on 
the  existence  of  "infant"  mortality  in  inanimate  equipment  as  there 
is  in  living  things.  This  writer  has  examined  much  data  collected 
from  many  sources,  has  reasoned  with  ae  much  logic  as  he  can  muster, 
and  has  talked  with  many  experienced  people  who  also  have  considered 
the  problem*  Re  concludes  that  true  infant  mortality  does  not 
exist  outside  the  realm  of  living  things. 

Consider  the  sea  turtle  which  crawls  from  the  ocean  and  l«ys  her 
eggs  in  the  sand  along  the  beach.  After  a  period  of  incubation,  many 
small  turtles  are  hatched  and  immediately  begin  to  make  their  way 
to  the  sea.  However,  sea  gulls  pounce  on  them  and  kill  many.  Only 
a  small  percent  ever  reach  the  comparative  safety  of  the  sea.  Of 
those  that  do,  some  are  eaten  by  fish.  Only  a  few  survive  to  reach 
maturity  and  die  from  old  age. 

The  above  example  illustrates  true  infant  mortality.  There  are 
many  such  examples.  In  fact,  all  living  things  experience  Infant 
mortality  due  to  natural  enemies,  disease,  and  pestilence.  But  what 
of  inanimate  objects  such  as  vacuum  tubes,  resistors,  capacitors, 
assemblies,  and  even  complete  systems?  Are  they  preyed  upon  by 
other  pieces  of  equipment?  do  assemblies  eat  components?  are  equip¬ 
ments  subject  to  diseases  which  are  more  severe  to  young  components 
than  to  older?  The  answer  should  be  obvious.  There  is  no  similarity 
between  inanimate  objects  and  living  things  as  related  to  infant 
mortality.  Only  in  regards  to  chance  and  wearout  is  there  is 
similarity. 

Here  it  seems  fitting  to  point  out  that  much  misunderstanding 
exists  relative  to  chance  and  wearout.  Far  too  much  credit  le  given 
to  failure  due  to  chance  when  in  reality  nearly  all  such  failures 
are  due  to  wearout.  In  a  field  where  there  is  much  variability  be- 
tveen  individual  components  we  would  expect  wearout  to  occur  in  such 
a  random  manner  as  to  appear  as  though  by  chance.  More  will  be  said 
about  this  later  in  the  paper. 

Figure  1  shows  the  hypothetical  model  distribution  upon  which 
PET  advocates  base  their  belief  that  production  equipment  can  be 
improved  by  debugging. 

Perhaps  the  purest  and  most  classical  example  of  PET  is  the 
naval  torpedo  proof  test  program  as  reported  by  Dr.  William  R,  Pabat, 
Jr.,  of  the  then  Bureau  of  Ordnance,  at  the  first  of  these  reliability 
symposia  ( NO L- Corona,  California,  October  195U)  in  hie  excellent 
paper  "Statistical  Planning  for  Ordnance  Proof  Testing." 


FIGURE  1.  MODEL  DISTRIBUTION  UNDERLYING  PET  CONCEPT 


Dr.  Pabst's  paper  began! 

"Torpedoes  are  in  many  respects  like  guided  missiles* 

They  are  complex,  automotive,  and  some  possess  intelligence  and 
self -guidance.  They  differ  in  many  nays,  too,  the  most  immediate 
differences  being  in  numbers  available,  in  the  quantity  of  past 
experience,  recoverability  after  test  run,  and  possibly  in  cost. 
Whether  torpedoes  will  differ  from  guided  mi ssiles  in  the  de¬ 
velopment  time  to  reach  a  state  of  high  reliability  may  depend  upon 
avoidance  of  some  of  tha  came  mistakes.  Because  of  these  simi¬ 
larities  and  this  desired  difference,  the  recent  planning  of  torpedo 
tests  may  guide,  either  by  a  warning  or  as  a  model,  for  those  in 
the  guided  missile  field. 

"Within  the  last  three  years,  the  system  for  proof 
testing  of  torpedoes  has  been  overhauled  and  placed  on  a  more 
systematic,  logical  and  statistical  basis.  Proof  tests  are  those 
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final  tests  given  torpedoes  before  they  are  released  as  serviceable 
torpedoes*  These  tests*  actually  run  in  the  eater*  are  considered 
to  provide  the  necessary  assurance  that  the  torpedo  will  perform 
as  needed  in  service  use.  In  the  past}  however*  because  of  the 
nature  of  torpedo  performance  and  because  of  the  failure  to 
interpret  teat  data  properly,  torpedo  testing  tended  to  provide  a 
fallacious  confidence  in  torpedo  reliability,  rather  than  the 
basis  for  improving  it, 

•The  reason  for  the  failure  of  proof  tests  to  provide 
the  necessary  action  leading  toward  reliability  resulted  from  the 
fact  that  Individual  torpedoes  were  run  and  rerun  until  they 
passed  the  test*  and  then  were  accepted  as  satisfactory.  It  was 
a  plausible  yet  erroneous  picture  that  the  faults  of  a  torpedo 
were  ehcvn  up  on  a  proof  run  and  corrected  before  the  next.  Thus 
it  was  also  erroneous  that  individual  torpedoes  improved  from 
these  adjustments  as  additional  runs  were  made.  To  an  outsider 
the  practice  of  repeating  alssile  systems  test  after  test  seems 
dangerously  similar. 


"The  old  system,  if  it  can  be  called  a  system*  can  be 
stated  simply.  If  the  torpedo  passed  the  test,  it  wae  sent  to 
the  fleet  as  a  ’passed  and  proved’  torpedo.  If  the  torpedo 
failed*  it  was  analyzed  for  cause  of  failure*  corrected  or 
adjusted  on  the  proof  range  or  at  the  factory  as  necessary  and 
rerun.  If  on  the  second  trial  it  met  standards,  it  was  considered 
’passed  and  proved’.  If  it  failed,  the  sequence  was  repeated  as 
many  times  as  required  for  the  torpedo  to  pass.  Some  torpedoes 
made  more  than  twenty  proof  runs  before  they  became  ’passed  and 
proved’.  One  Mark  of  torpedoes  required  on  the  average  of  10.5 
runs  per  torpedo  over  a  series  of  years.”  Figure  2  shows  the  failure 
rate  for  one  Mark  of  some  2600  torpedoes.  The  points  have  been 
connected  in  order  to  show  ary  trend  that  might  exist. 
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FIGURE  2  TORPEDO  FAILURE  RATE 
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Note  the  randomness  of  the  curve.  Definitely,  no  infant,  mortality 
is  present.  - 

Quoting  further  from  the  paperi 

"Proof  test  runs  over  a  aeries  of  years  were  analyzed. 

Of  several  thousand  torpedoes,  approximately  twenty  percent  passed 
on  the  first  run.  Of  the  remainder  tested  on  the  second  run,  again 
approximately  twenty  percent  passed.  On  the  third,  fourth  and 
successive  runs  of  the  diminishing  balance,  approximately  the  seme 
percentage  with  allowance  for. sampling  variation  continued.  The 
result  was  about  the  same  as  if  one  started  with  many  five  sided 
dies  and  counted  ae  satisfactory  in  succession  those  dies  on  which 
aces  appeared.  If  torpedo  performance  actually  improved  after  the 
successive  adjustments  and  overhauls,  the  percentage  score  passing 
would  have  improved  with  each  run.  Contrary-wise  If  the  test  had  sep¬ 
arated  out  the  bad  torpedoes  from  the  good,  the  percentage  score  would 
have  decreased  with  each  run*  The  consistently  even  score  from  run 
to  run  simply  suggested  that  passing  the  test  was  a  chance  phenomena 
and  that  the  adjustments  and  changes  made  in  torpedoes  from  run  to 
run  were  simply  offsetting. 

"Do  not  at  this  point  be  alarmed  about  the  twenty  percent 
score.  This  twenty  percent  measures  as  much  the  artificiality  of 
the  old  technical  proof  specifications  on  the  individual  runs  as  it 
does  the  performance  of  the  torpedoes  themselves.  As  will  be  shown, 
the  specifications  based  upon  original  hope  needed  to  be  tempered 
by  experience. 

"In  view  of  the  consistent  percentage  of  torpedoes  passing 
in  the  original  proof  test,  it  was  a  statistical  prediction  that 
"passed  and  proved"  torpedoes  would  repeat  about  the  same  per¬ 
centage  If  run  again.  Therefore,  a  special  test  was  carried  out 
in  which  five  passed  and  proved  torpedoes  were  tested  five  times 
each  along  with  five  pre range  torpedoes.  Prerange  torpedoes  are 
those  that  had  not  previously  been  water  tested.  These  comparative 
te6ts  wire  statistically  designed  so  that  all  tan  torpedoes  were 
given  the  same  treatment  and  all  run  on  the  same  days  approximately 
two  veek6  apart.  In  overall  results,  the  prerange  torpedoes  scored 
seven  satisfactory  runs  in  the  twenty-five  trials  in  contrast  to 
three  eatisfactory  runs  for  the  "passed  and  proved"  torpedoes. 

It  was  probably  accidental  that  the  prerange  torpedoes  scored 
slightly  better,  for  the  difference  did  not  meet  tests  of  statistical 
significance.  In  specific  characteristics  of  depth,  deflection,  and 
speed  the  two  groups  were  very  similar.  The  results  of  this  test 
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confirmed  the  prediction  that  the  single  satisfactory  runs  of  the 
'passed  and  proved'  torpedoes  did  not  make  them  more  reliable  with 
respect  to  their  future  performance, 

‘’These  results  also  supported  the  contention  that  passing 
cne  proof  run  was  largely  a  matter  oi  chance.  Improved  reliability 
hao  to  be  sought  in  increasing  this  chance,  that  is,  through  in¬ 
creasing  the  proportion  of  satisfactory  runs,  thus  through  higher 
quality  levels."  End  of  quote, 

A  striking  -.imilarity  exists  between  the  torpedo  proof  tests 
and  the  application  of  PET  to  missile  hardware.  In  the  proof 
tests,  no  vain  attempt  had  to  be  made  to  simulate  environment,  for 
theso  tests  were  run  in  water,  the  same  as  in  use  and  the  torpedoes 
were  recovered;  however,  in  the  case  of  guided  missiles,  manor 
attempts  at  simulation  have  been  made,  with  varying  degrees  of 
sophistication. 

The  results  of  this  above  cited  designed  experiment  clearly 
indicated  that  no  early  high  failure  rate  existed,  followed  by  a 
"long  low  stretch"  of  constant  failure 3.  It  was  demonstrated 
that  passing  such  a  test  (as  in  PET)  was  but  a  chance  phenomena. 

Yesterday  afternoon,  R.  D,  Sinter  of  the  Bureau  of  Weapons,  in 
his  paper,  "Bullpup,  A  Round  of  Aranunition",  said  the  following: 

"BULLPUP  specifications  have  aiwaya  required  that  the 
missile  meA  a  particular  reliability  goal.  This  goal  has  been 
continually  increased  during  development,  evaluation,  and  now 
production.  All  of  these  specifications  recognized  that  continued 
testing  of  the  missile  during  'ready  service'  stowage  would  reduce 
the  total  missile  reliability.  The  specifications  allowed  a 
reduction  in  reliability  after  each  test.  That  reliability  would 
be  reduced  due  to  continued  testing  was  proven  during  the  Navy 
Technical  Evaluation.  A  block  of  30  missiles  was  subjected  to 
repeated  operational  tests.  None  failed  on  the  first  or  second 
tests.  Failures  began  to  appear  on  the  third  and  subsequent  tests. 
Testing  missiles  in  an  attempt  to  prove  them  reacfcr  was  actually 
causing  them  to  fail,"  Figure  3  shows  the  failure  rate  curve. 

should  be  noted  that  the  sample  size  varied  from  30  missiles 
on  the  first  test  to  1  on  the  eighth.  That  part  of  the  curve 
beyond  test  number  6  does  not  necessarily  present  a  true  picture. 
Had  the  sample  been  larger,  it  is  reasonable  to  believe  that  the 
shape  of  the  curve  might  be  changed  9omevhat.  However,  it  is 
interesting  to  compare  this  curve  with  that  of  Figure  9.  The 
reason  for  the  failure  rate  to  fall-off  to  zero  percent  is  due  to 
there  being  virtually  no  samples  remaining,  as  is  the  case  in 
human  mortality  beyond  age  100. 
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FIGURE  3.  BULLPUP  FAILURE  RATE 


This  too,  clearly  Indicates  the  absence  of  an  early  failure 
rate  followed  by  the  "long  low  stretch".  In  fact,  just  the 
opposite  is  demonstrated. 

In  December  195U,  General  Medaris,  at  that  time  Chief  of 
Industrial  Division,  Office,  Chief  of  Ordnanoe,  directed  that  the 
Corporal  PET  program  be  stopped.  Simultaneous  with  this  decision. 
Firestone  Tire  and  Rubber  Company,  the  Corporal  production  contractor 
published  an  analysis  of  6  months  PET  results  of  11  electronic 
assemblies.  These  tests  consisted  of  subjecting  all  production  of 
the  11  electronic  assemblies  to  extended  periods  of  vibration.  If 
failures  occurred,  the  equipment  was  repaired  and  retested  in  a 
manner  similar  to  the  torpedo  proof  test  program.  The  results  are 
shown  in  Figures  U,  5  and  6.  (Sample  sizes  on  first  shake  varied 
from  1?  to  119,  decreasing  in  number  as  subsequent  shake  tests 
were  conducted). 
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FIGURE  6.  CORPORAL  ASSEMBLY  FAILURE  RATE 


It  is  seen  that  there  is  no  "early"  failure  pattern  exhibited, 
but  rather  a  random  one.  This  is  further  proof  of  Dr.  Pabst's 
contention  that  passing  such  tests  is  merely  a  matter  of  chance, 

A  further  study  of  Corporal  PET  was  made  by  R.  P.  Henry,  of 
Firestone,  In  a  report  dated  30  December  195L,  "Analysis  of 
Vibration  Results".  This  analysis  was  of  PET  results  on  eight 
electronic  "black  boxes"  chosen  because  they  were  thought  to  be 
"intrinsically  weak".  In  this  report  it  vas_said,  "....to  continue 
to  shake  packages  as  a  routine  production  test  to  discover  inherent 
weaknesses  may  perpetrate  a  system  which  introduces  as  mary  failures 
as  it  discovers." 

The  purpose  of  this  analysis  by  Mr.  Henry  was  stated  as  follows: 
"The  analysis  in  this  report  attempts  to  assess  this  possibility 
/Introducing  as  many  failures  as  it  discovers/  with  a  simple  but 
statistically  rigorous  test  known  as  the  contingency- type  test 
making  use  of  the  chi-square  distribution.  It  will  answer  the 
questions:  (1)  Are  units  failing  at  the  same  rate  in  second  'shake' 
as  they  are  in  the  first?  (2)  Do  those  units  which  pass  first 
'shake'  but  were  reshaken  for  some  reason  fail  at  the  same  rate  as 
those  which  failed  first  shake  and  had  to  undergo  additional  testing? 


It  was  concluded:  "The  evidence  from  these  eight  packages 

all  point  in  this  direction . It  is  hypothesized  that  there 

is  no  real  difference  in  shake  failure  rates  from  one  shake  to 
the  next.'  Only  in  the  case  of  2  of  the  8  assemblies  tested  is 
it  possible  to  reject  this  idea;  and,  in  these  cases,  the  failure 
rates  are -actually  worse.  In  spite  of  this,  it  is  possible  to 
pass  a  unit  through  vibration  if  enough  tries  are  made.  In  fact, 
all  units  in  the  plant  have  passed  a  vibration  test  at  one  time 
or  another.  However,  this  analysis  casts  considerably  doubt  upon 
the  ability  of  a  unit  to  pass  a  subsequent  vibration  test  (or 
flight)  with  any  greater  probability  than  the  previous  one,  which 
of  course,  is  no  net  improvement." 

This  is  further  proof  that  infant  mortality  is  not  present  in 
inanimate  objects.  Con  it  be  that  infant  mortality  applies  only 
to  infants? 

At  the  time  the  decision  was  made  to  stop  the  Corporal  PET 
program,  it  is  noteworthy  that  the  RfcD  contractor  (who  had 
written  the  PET  specifications)  steadfastly  maintained  that  the 
Corporal  reliability  would  be  degraded  seriously  if  the  PET 
were  not  continued.  Figure  7  shows  the  reliability  status  of  the 
Corporal  during  this  period. 


MISSILE  SERIAL  NUMBER 


FIGURE  7  CORPORAL  RELIABILITY  CURVE 


A  twenty-round  soring  are rage  (gone  irregularities  in  the  curve 
hare  been  smoothed  oat  to  simplify  the  diagram)  of  approximately 
100  rounds  of  Missiles  with  electronic  hardware  which  was  subjected 
to  PST  Is  shown  in  relation  to  approximately  the  same  number  of 
••is tiles  haring  no  hardwara  so  mistreated. 

It  can  ba  seen  that  thare  is  no  startling  change  in  reliability 
status  which  can  be  related  to  PST.  A  gradual  increase  was  to  be 
expected  due  to  normal  system  improvement  caused  by  correct  ire  actions 
resulting  from  field  experience,  combined  with  troop  imprormront  in 
operational  skills. 

In  another  guided  missels  program  in  which  100)1  vibration 
testing  Is  being  conducted,  the  contractor  lists  the  following 
items  as  the  kind  of  failures  or  defects  he  ie  finding! 

Loose  nuts  and  bolts 
Cold  solder  joints 

Insulation  wearing  through  from  rubbing 

Broken  capacitor  leads 

Capacitor  shorting 

Microphonics  on  trlmpots 

Hicrophonic  transistors 

Intermittent  relays 

Broken  wires 

Broken  mountings 

Cracked  transistors 

Loose  contacts 

Micro syn  gear  retainer 

Poor  mechanical  fit  and  looseness 

Defective  pot  wipers 

Snap  ring  failures 

He  concluded  that  these  defects  "would  not  hare  been  discovered 
during  normal  manufacturing  Inspection."  He  also  made  the  following 
"rediscovery":  "Manufacturing  failures  appeared  to  be  completely 
random." 

I  think  we  can  assume  safely,  that  if  the  contractor  states 
that  these  failures  and  defects"  would  not  hare  been  discovered 
during  normal  manufacturing  inspection",  either  his  "normal" 
inspection  is  no  good,  or  else  these  defects  were  not  present  at 
the  time  of  inspection,  and  were  therefore,  the  direct  result  of 
the  PET.  How  else  does  insulation  wear  through  from  rubbing? 
surely  not  by  sitting  at  rest  undergoing  normal  inspection. 

In  his  paper  "Production  Environmental  Testing"  (November 
1957),  Robert  Lusser  showed  that  the  "Mortality  curve"  is  shaped 
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by  "fore*  of  Mortality".  This  la  shown  in  Figure  8.  Note  that  the 
force  of  aortal lty  must  experience  a  decrease  with  tiwj  in  order 
for  the  Mortality  to  exhibit  a  like  decrease. 


FIOURX  8.  HUMAN  MORTALITY  DISTRIBUTION 


Thia  le  the  case  with  turtles,  with  all  living  creatures,  and 
with  human  beings  where  the  force  of  mortality  decreases  rapidly 
froa  time  of  birth  until  age  5  or  6,  then  is  fairly  constant  until 
vs  reach  age  U5,  thereafter  rising  steeply  until  no  samples  remain, 
all  having  failed  Mostly  froe  wearout. 

Consider  now  the  case  of  the  "force  of  mortality"  in  electronic 
equipment.  Is  there  any  similarity  to  human  behavior?  Is  there 
any  cause  for  a  decrease  in  "force  of  mortality"  during  an  early 
pe.  iod  of  time  in  which  equipment  can  be  improved  by  "debugging"? 

The  answer  is  an  emphatic  no. 

In  complex  equipment,  wear-out  begins  immediately  in  same 
parts,  such  that  the  overall  force  of  mortality  immediately  begins 
to  rise,  however  slight  may  be  indicated,  resulting  In  e  gradual 
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increase  of  mortality.  This  is  shown  in  Figure  9.  Note  the 
similarity  to  ths  data  presented  by  Ointsr  on  "Bullpup",  as 
was  shown  in  Figure  3- 


FIGURE  9.  EQUIPMENT  MORTAL ITT  DISTRIBUTION 

ARINC  Research  Corporation  reported  the  results  of  a  study 
made  of  incoming  inspection  and  screening  techniques  of  vacuus 
tubes  utilized  by  three  equipment  manufacturers.  This  studf, 
"Evaluation  of  Incoming  Inspection  and  Selection  Procedures  for 
Electron  Tubes",  dated  1*  April  1958,  revealed  some  interesting 
things. 

In  the  Foreword  the  following  is  written: 

"The  effectiveness  of  special  incoming- inspection  and 
selection  procedures  as  'screening  agents'  for  reliability  assurance 
is  discussed.  Three  electron-tube  inspection  processes,  generally 
representative  of  those  being  used  by  the  ©ouipment  industry  to 
weed  out  potential  tube  failures,  are  evaluated  in  detail.  On 
the  basis  of  these  data,  a  general  evaluation  of  electron-tube 
inspection  procedures  is  formulated. 

"Although  the  report  is  restricted  to  a  discussion 
of  Inspection  procedures  related  to  electron  tubes,  it  is 
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belie.  .:d  that  the  finding#  are  applicable  to  many  of  the  special 
proo  u,*es  used  to  inspect  and  select  other  types  of  electronic 
parts  and  components.. , ." 

"Any  one  of  several  different  methods  could  have  been 
used  to  evaluate  the  incoming-inspection  procedures  described  in 
this  report.  However,  ARINC  strongly  believes  that  approaches 
other  than  those  employed  would  have  led  to  the  same  conclusions,.,," 
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From  the  Summary  the  following  is  extracted! 


"The  equipment  industry  has  sought  to  complement  the 
efforts  toward  basic  tube  improvement  by  establishing  incoming- 
inspection  processes  designed  to  screen  defective  tubes  from 
new  lots  as  thsy  wers  purchased.,..  The  number  of  "defective" 
tubes  rejected  from  each  tube  lot  ran  high  —  as  did  the  cost  of 
acceptable  tubas. 

"Particularly  because  of  the  cost  factor,  it  became 
deal /able  to  evaluate  the  more  typical  incoming-inspection  processes 
in  'rder  to  single  out  those  whose  contribution  to  tube  reliability 
was  sufficient  to  merit  consideration  for  adoption  in  military 
specifications..,,  four  controlled  tests  were  conducted  to 
evaluate  three  different  Inspection  processes. 

"These  tests  revealea  no  significant  unproven*. nt  either 
in  tube  reliability  or  in  equipment  reliability  a  result  of 
the  special  tube-processing  techniques  invests  .  The 
equipment  manufacturers  whose  inspection  tests  . .  chosen  for 
evaluation  have  eliminated  these  processes  from  their  incoming- 
inspection  procedures.  Other  equipment  manufacturers  who  are 
aware  of  the  findings  have  talo'-n  similar  action." 

The  tests  reported  consisted  of  various  combinations  of 
visual  -  microscopic,  vibration-noise,  1-ray,  polariscopic, 
and  thermal  shock.  Generally,  a  batch  of  tubes' was  divided  Into 
two  groups:  a  control  group  and  a  test  group.  The  control  group 
was  installed  in  the  equij  nent  and  operated  until  the  removal 
rate  (as  a  measure  of  reliability^  was  determined.  At  the  same 
time,  the  test,  group  vas  run  through  the  equipment  manufacturers 
screening-inspection  process  and  were  thus  separated  into  2  groups! 
those  which  passed  the  test,  and  those  which  failed. 

The  in  turn,  were  also  Installed  in  the  equipment  and 
operated  until  the  removal  rate  was  determined.-  In  all  cases, 
no  significant  difference  was  found  in  the  removal  rates  of  the 
control  group,  the  "good"  tubes  and  the  "bad"  ones. 
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In  the  Proceedings  of  the  IRE,  August  1956,  page  1073,  the 
following  is  said  under  the  topic,  "Reliability  Factors  for  Ground 
Equipment"  i 

"It  is  now  recognised  that  tube  lellures  are  most  likely 
to  occur  during  their  early  history,  say,  the  first  50  or  100  hours. 
Like  human  beings,  whose  mortality  is  greatest  during  their  child¬ 
hood,  if  tubes  get  over  this  critical  period,  they  are  likely  to 
have  a  normal  life  expectancy.  To  avoid  these  'quickie*  troubles 
it  is  becoming  customary  to  'bum  in'  tubes  or  to  age  them  before 
they  are  accepted  or  are  permitted  to  go  into  the  equipment.  Alter 
this  period,  failures  seem  to  follow  an  exponential  pattern;  in  a 
given  time  a  given  percentage  of  those  still  in  service  must  be 
replaced." 


Mr.  C.  Ft.  Knight,  Director,  AR1NG  Research  Corporation, 
responded  with  the  following  comment: 

"This  is  not  so  much  an  error  as  reflection  of  the  date 
of  the  material.  This  statement  was  believed  to  be  true  rather 
generally  up  through  1953.  .Growing  masses  of  data  since  that  time, 
however,  have  quite  firmly  established  that  in  receiving  type 
electron  tubes  this  'infant  mortality'  is  no  longer  apparent  and 
that  the  probability  distribution  of  time  to  failure  in  receiving 
tubes  is  most  frequently  Gaussian.  The  unnecessary  'burning  in' 
of  tubes  by  equipment  manufacturers  and  the  attendant  economic 
waste  is  a  primary  concern  of  military  procurement  agencies  at  this 
time." 

F*urthermore,  D.  S.  Peck,  Bell  Telephone  Laboratories,  in  his 
paper,  "A  Mesa  Transistor  Reliability  Program"  presented  yesterday 
afternoon,  noted  that  while  undergoing  life  tests,  "transistors 
experienced  a  constant  failure  rate", 

D.  B.  Christian,  statistical  Analysist,  Bend lx  Products 
Division  -  Missiles,  wrote  the  following  in  the  abstract  of  his 
paper,  "Qcani  nation  of  Reliability  Theory'  and  Practice"  presented 
at  this  symposium  yesterday  morning:  "The  use  of  the  infant 
mortality  concept  give  •  a  fake  impression  that  things  will  get 
better  as  faulty  components  are  weeded  out.  But  what  assurance  do 
we  have  that  the  replaced  parts  are  better?  The  author  has  been 
told  that  testing  de-bugs  electrical  systems  so  that  more  testing 
improves  the  product.  This  paper  presents  facts  that  failures  are 
a  function  of  time  and  the  failure  rate  is  essentially  constant 
throughout  the  assembly  test  period.  This  does  not  indicate 
infant  mortality." 
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Now,  let's  examine  PET  from  the  point  of  logic.  In 
discussing  this,  I  would  like  to  take  as  example,  wire  wound 
resistors.  1  read  in  a  reliability  report  published  at  a  military 
installation  3  years  ago  a  discussion  of  such  an  example.  It  was 
r»»«oned  that  wire  wound  resistors  failed  due  to  "nicks"  in  the 
wire  which  were  produced  during  manufacture.  It  was  assumed  that 
the  time  to  failure  under  use  conditions  was  proportional  to  the 
sise  of  the  "nick"  in  the  wire.  It  was  concluded  that  by  burning* 
in  these  resistors  for  a  prescribed  period  of  time,  the  resistors 
with  defects  (or  nicks)  would  fail,  thus,  leaving  the  "good" 
resistors.  The  fallacy  of  this  reasoning  lies  in  the  assumption 
that  resistors  either  have  "nicks"  or  do  not  have  "nicks".  The 
truth  of  the  matter  is,  however,  that  nicks  produced  by  a 
manufacturing  process  would  be  distributed  from  the  largest  (which 
night  be  visible  to  the  naked  eye),  to  the  smallest  (which  might 
be  detected  only  with  a  powerful  microscope). 

So,  when  does  one  stop  the  "burning -in"  in  order  to  separate 
bad  from  good?  No  matter  what  time  is  chosen  arbitrarily,  always 
there  is  the  next  smaller  nick  which  will  cause  the  resistor  to 
fall  In  the  next  Interval  of  time.  That  Interval  of  time  might 
well  be  the  flight  period  under  an  actual  engagement. 

It  rauart  be  concluded  that  screening  techniques,  so  closely 
related  to  PET,  are  worthless  unless  parts  screened  actually 
are  composed  of  a  blmodal  population,  consisting  of  only  "good" 
and  "bad"  parts  with  no  inbetween. 

I  often  use  the  example  of  eggs.  Were  I  given  a  basket  of 
eggs,  some  of  which  were  very  fresh,  and  the  remainder  very  rotten, 
it  would  be  possible  to  design  a  test  whereby  the  good  eggs  could 
be  separated  from  the  bad  with  100J  certaintyj  but,  if  the  eggs 
were  randomly  distributed  from  very  good  to  very  bad,  it  would  be 
impossible  to  separate  all  of  the  good  ones  from  the  bad.  For  by 
what  yardstick  would  one  measure  a  good  egg?  or  a  bad  one?  There 
are  manj  people  In  the  vorld  today  who  eat  eggs  which  you  and  I 
would  reject ! 

In  the  November  195!>  issue  of  "Scientific  American"  there 
appeared  an  interesting  article,  "The  Growth  of  Nerve  Circuits", 
by  R.  V.  Sperry,  Ilixon  Professor  of  Psychobiology  at  the  California 
Institute  of  Technology.  In  the  article,  the  author  discusses  the 
effect  on  humans  and  animals  of  nerve  injuries  and  the  possibility 
of  restoring  normal  nerve  function  through  surgical  means  and  by 
training  and  re-education. 
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The  author  telle  how,  scraetinea,  to  prevent  Atrophy  of  the 
facial  erase lea  when  an  injured  facial  nerve  fails  to  regenerate, 
surgeons  will  connect  the  degenerated  facial  muscles  to  a  nearby 
healtly  nerve,  such  as  the  motor  nerve  of  the  shoulder  muscle. 

"Naturally,  the  primary  concern  of  the  patient  in  such  cases 
is  whether  or  not  normal  function  can  be  restored.  If  the  symptoets 
do  not  clear  up  spontaneously,  can  they  be  corrected  by  training 
and  re-education?  Not  so  long  ago,  the  reply  to  such  questions 
was  a  confident  'yes'.  For  most  of  the  present  century  in¬ 
vestigators  and  physicians  were  agreed  that  the  central  nervous 
system  is  plastic  enough  so  that  a ny  muscle  nerve  might  be 
reconnected  to  ary  other  muscle  nerve  with  good  functional  success." 

"During  the  past  15  years,  however,  scientific  and  medical 
opinion  have  undergone  a  major  shift,  amounting  to  an  almost 
complete  about-face." 

"The  evidence  for  this  /about -f«ct7,  which  cones  from  new 
experiments  and  from  exacting  clinical  observations,  is  so  per¬ 
suasive  that  it  is  difficult  to  understand  how  the  opposite  view 
could  have  prevailed  so  long.  It  appears  that  most  of  the  earlier 
reports  of  the  high  functional  plasticity  of  the  nervous  system 
will  go  down  in  the  record  as  unfortunate  examples  of  how  an 
erroneous  medical  or  scientific  opinion,  one#  implanted,  can  snow¬ 
ball  until  it  biases  experimental  observations  and  crushes  dissenting 
interpretations." 

Can  it  be  that  the  same  is  true  of  PET?  that  PET  is  another 
example  of  an  erroneous  scientific  opinion,  which,  once  implanted, 
has  snowballed  until  it  has  biased  experimental  observations  and 
crushed  dissenting  interpretations? 

The  management  of  a  contractor  which  is  producing  hardware 
for  the  government  is  told  by  his  technical  personnel  that  a  certain 
screening  technique,  vibration  test,  or  PET,  is  separating  good 
product  from  bad  and  is  thus  a  useful  quality  control  technique.  The 
management  accepts  this  as  true,  and  in  fact  is  happy  in  this  belief 
that  such  a  test  Is  useful  in  improving  his  product  and  in  providing 
assurance  that  the  pro  uct  he  is  delivering  meets  specifications 
and  is  good. 

The  fallacy  arises  from  the  fact  that  a  production  contractor 
is  paid  only  for  the  hardware  which  he  delivers  from  hie  loading 
dock.  He  therefore  sees  no  need  to  evaluate  realistically  the 
results  of  the  screening  test  or  PET  being  used.  He  continues  to 
ship  hardware  to  the  government  which  he  sincerely  believes  is 


better  because  of  the  test.  What  he  doesn't  know  Is  that  in  all 
cases  where  PET  results  hare  been  evaluated,  it  has  been  demon¬ 
strated  that  no  product  Improvement  has  resulted.  That  is,  the 
product  which  passed  the  PET  screening  was  no  better  than  that 
which  was  given  no  such  test. 

The  argument  was  used  by  one  contractor  that  PET  provided 
the  only  source  of  failure  data  he  had.  He  was  conducting  a 
mild  test  to  failure  on  a  100$  basis  and  was  then  delivering  the 
used  merchandise  to  the  government  1 

The  govement  must  move  to  eliminate  foolish,  wasteful,  and 
time  consuming  practices  such  as  PET.  It  must  insist  that 
contractors  determine  appropriate  environmental  conditions,  test 
to  failure  adequate  samples  of  RSd)  (and  later  production)  hardware 
in  these  environments,  and  demonstrate  the  existence  of  adequate 
safety  margins.  Furthermore,  it  must  be  willing  to  support  these 
activities  with  adequate  funds  and  proper  schedules  if  it  expects 
to  receive  reliable  guided  missile  systems  from  its  contractors. 


THE  ATLAS  RELIABILITY  TEST  PROGRAM  - 
INCREASED  RELIABILITY  RESULTING  PROM 
COMBINED  ENVIRONMENTAL  TESTING 

Ry  C.  C.  Campbell 
Senior  Reliability  Engineer 
Comrelr  Astronaut  ice 

L  INTRODUCTION 

The  high  environmental  severity  levels  in  which  today's  complex  mis¬ 
sile  systems  must  operate  create  a  serious  reliability  problem*  The  re¬ 
cord  of  miaaile  flight  failures  during  the  early  developmental  stage 
stresses  the  magnitude  of  the  problem*  fejor  companies  in  the  missile 
Industry,  fully  aware  of  the  stakes  Involved,  are  an  all-out 

effort  to  solve  the  reliability  problem*  However,  since  no  single 
methodology  has  been  developed  that  can  be  utilised  to  insure  the  relia¬ 
bility  of  each  and  every  system,  each  company  has  approached  the  relia¬ 
bility  problem  from  a  different  angle*  This  paper  presents  the  relia¬ 
bility  approach  taken  by  Convair- Astronautics  for  the  Atlas  Missile 
system*  In  addition,  the  results  achieved  during  the  first  four  phases 
of  the  Atlas  Reliability  "Search  for  Critical  Weakness*  teat  program  are 
presented* 

The  "Search  for  Critical  Weaknese"  test  program  is  based  on  the  con¬ 
cept  of  early  detection  of  component  weaknesses  in  design,  materials, 
and  fabricating  techniques  so  that  corrective  action  can  be  implemented 
before  operational  failure  occurs*  In  pursuing  these  objectives,  the 
concentration  of  effort  has  been  on  the  improvement  of  component  relia¬ 
bility.  Weaknesses  were  determined  by  exposing  ndssileborne  components 
to  simulated  flight  conditions  in  the  environmental  laboratory,  with  the 
realistic  features  of  combined  environmental  testing* 

The  reliability  test  program  for  the  Atlas  missile  system  is  admin¬ 
istered  by  the  Reliability  organisation  within  Convair-Astronautics.  In 
contrast  to  this, many  other  prime  contractors  delegate  the  res po ns i billy 
for  reliability  tests  to  the  manufacturers  from  whom  components  are 
being  obtained.  There  are  several  advantages  In  having  a  group  within 
the  prime  contractor's  organization  responsible  for  all  reliability 
tests.  Some  of  the  advantages  are*  1)  close  liaison  is  maintained 
with  design  groupe,  thus  providing  for  a  coordinated  effort  in  determin¬ 
ing  adequate  solutions  for  detected  component  weaknesses)  2)  close 
liaison  is  maintained  with  the  theoretical  groups  (Aerophysics, Thermo¬ 
dynamics,  and  Dynamics)  for  determining  realistic  environmental  condi¬ 
tions  j  3)  close  liaison  13  maintained  with  the  flight  evaluation  groups 
for  feed-back  information  from  actual  missile  flights)  1*)  test  require¬ 
ments  are  standardized  for  all  components,  resulting  in  a  valid  compara¬ 
tive  assessment  of  component  reliability;  5)  an  independent  evaluation 
is  made  of  all  items  tested  -  thus,  all  biases  are  eliminated  from  test 


results  so d  isolated  from  tbe  influence  of  the  designer  or  the  manu¬ 
facturer.  The  general  testing  philosophy  adopted  for  the  "Sesrch  for 
Critical  Weakness  Program"  is  suanarized  as  follows t 


1*  Component  Types  Tested 

All  missileboroe  components  which  fall  within  Convair  Astronau¬ 
tics'  responsibility  and  possibly  have  a  critical  effect  upon  an 
Atlas  mission  were  selected  for  tests.  Up  to  this  time  more  than 
300  component  types  hare  completed  reliability  tests.  These  com¬ 
ponents  included  electrical  components  such  as  battery  packages, 
inverters,  transponders j  hydraulic,  pneumatic  and  propulsion 
Items  such  as  staging  valves,  reservoirs,  end  accumulators )  and 
electro-mechanical  items  such  as  servo  valves,  separation  de¬ 
vices,  programmers  and  deatruct  systems. 

2.  Sample  Size 

The  number  of  specimens  to  be  tested  was  determined  by  the  total 
number  of  each  component  used  per  missile,  and  by  its  relative 
importance  to  the  overall  system.  Short-life  or  one-shot  type 
components  such  as  primary  batteries  and  explosive  devices 
necessitated  the  selection  of  additional  specimens.  Program 
budgeting  was  established  at  the  outset  on  the  basis  of  five 
equivalent  ahipsets  for  Series  A  Atlas  Missiles, three  for  Series 
6,  three  for  Series  C,  five  for  Series  D  and  six  for  Series  E. 
This  sample  size  has  provided  greater  confidence  in  the  test  re¬ 
sults  and  an  indication  of  production  and  performance  variabili¬ 
ty.  Several  instances  have  occurred  where  only  one  out  of  10  or 
20  specimens  failed  during  the  program.  Por  example,  one  Hydrau¬ 
lic  tank  out  of  10  tested,  burst  during  Reliability  testa.  After 
revealing  a  structural  weakness  in  the  component, it  was  strength¬ 
ened, leading  to  a  more  reliable  unit  (See  Figure  1).  The  chances 
of  revealing  such  a  weakness  would  have  been  small  if  only  one 
sample  had  been  tested* 

3.  Selection  of  Test  Specimens 

It  is  essential  that  the  test  specimens  are  identical  to  flight 
articles  and  that  the  manufacturer  is  not  aware  which  items  are 
to  be  reliability  tes;ed,  For  these  reasons  the  teat  specimens 
are  selected  at  random  from  production  stock. 

U.  Environmental  Conditions 


All  equipment  is  exposed  to  simulated  operational  conditions  in 
the  environmental  laboratory.  In  order  to  obtain  information  in 
the  minimum  time  and  to  minimize  the  effects  of  fatigue, only  the 
most  adverse  environments  are  selected  for  test  conditions.  In 
most  cases  these  environments  are  sustained  acceleration,  high 
and  low  temperature,  altitude  and  vibration.  During  Series  A 
testing,  these  environments  were  applied  singly.  During  all 
series  thereafter,  temperature ,  altitude  and  vibration  were  com¬ 
bined  with  the  introduction  of  programmed  radiant  heat  where 
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applicable.  While  experiencing  these  environments,  the  equip¬ 
ment  was  in  an  operating  condition  representing  its  tine  se¬ 
quencing  and  end-use  in  flight* 

5*  Rnvironnental  Severity  Levcla 

Although  equipment  may  show  adequate  performance  at  the  design 
level  in  qualification  testa,  this  does  not  necessarily  prove 
that  it  has  attained  the  high  reliability  required  for  the  Mis¬ 
sile  system*  Consequently,  all  specimens  are  tested  at  the  de¬ 
sign  level  of  environmental  intensity  and  then  at  two  levels  be¬ 
yond  this  value*  Testing  beyond  the  design  requirements  is  dons 
to  establish  a  margin  of  safety  for  each  component  type*  This 
safety  margin  nay  then  be  used  aa  a  yardstick  of  equipment  relia¬ 
bility  for  comparative  purposes.  Testing  at  more  than  one  level 
also  is  important  in  the  event  that  actual  flight  environmental 
severities  differ  from  the  expected,  or  in  the  event  that  a 
component  is  relocated*  In  this  case,  the  teat  results  permit 
us,  without  further  testing,  to  establish  a  new  safety  margin 
and  thus  reassess  reliability*  Figure  2  illustrates  a  failure 
incurred  by  a  heliua  staging  valve  during  the  third  level  of 
vibration  intensity.  Subsequent  examination  disclosed  that  the 
valve  had  been  weakened  by  undercutting  in  the  ares  of  s  sharp 
thread  root*  The  weakness  was  easily  corrected  by  eliminating 
the  undercutting  process,  thus  strengthening  the  valve  consider¬ 
ably.  ThJs  weakness  would  not  have  been  revealed  if  testa  had 
been  conducted  at  only  the  design  level  requirements* 

6*  Test  Results 

When  the  final  test  results  and  aqy  interim  results  are  received 
by  the  Reliability  Organisation,  they  are  reviewed  in  collabora¬ 
tion  with  the  cognizant  design  group  to  provide  an  initial  eva¬ 
luation  of  the  equipment's  reliability*  Follow-up  action  to 
eliminate  any  reliability  problem  is  coordinated  by  the  Relia¬ 
bility  Organisation.  A  written  summary,  known  as  a  Reliability 
Test  Evaluation,  is  prepared  by  the  Reliability  Organisation  and 
submitted, together  with  all  aaeociated  reports,  to  the  cognisant 
design  group. 

The  Reliability  Organisation  meanwhile,  conducts  the  necassary 
statistical  analyses  required  for  a  final  reliability  evaluation* 
These  include:  a)  an  investigation  of  variability  in  performance 
data  as  a  function  of  the  environmental  severities;  b)  a  study 
of  the  relationship  between  safety  margin  and  equipment  relia¬ 
bility;  and,  c)  an  evaluation  of  comparative  reliability.  After 
the  final  analysis , component  performance  is  classified  under  one 
of  the  three  following  categories:  1)  satisfactory,  2)  out-of- 
tolerance  failure,  and,  3)  catastrophic  failure.  Within  these 
three  classifications,  each  component  is  further  claaslfl.ed 
according  to  the  severity  level  at  which  *t  failed  and  the  number 
of  failures.  This  listing  of  components  then  served  as  an  indi¬ 
cation  of  the  relative  reliability  of  each  component  listedjfrom 
the  most  reliable  to  the  least  reliable* 
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B.  TYPICAL FAILURES 


One  factor  leading  to  the  reTelatlon  of  many  weaknesses  was  the 
application  of  combined  environments  while  the  specimens  were  operating* 
Several  structural  type  weaknesses  were  revealed  such  as  faulty  welds, 
e>>£. tings,  flanges  and  brackets*  Leakage  around  the  various  types  of 
seals  was  prevalent  at  low  temperature-vibration.  Diaphragm  damage  and 
aluminum  strength  created  a  problem  at  high  temperature-vibration*  Arc- 
over  led  to  one  serious  failure  at  combined  temperature-vibration-alti¬ 
tude,  Electronic  equipment,  in  general,  was  weak  with  respect  to  broken 
leads,  solder  joints,  vacuum  tubes,  and  wiring  interference  pro  blase. 
Electrolyte  leaked  from  all  battery  canisters*  Rotating  equipment  was 
particularly  susceptible  to  sustained  acceleration  testing*  Three  such 
weaknesses  appeared  during  the  test  program  1)  the  inverter  armature 
slipped  on  its  shaft  severing  the  output  leads}  2)  the  Asusa  transponder 
fan  blades  were  damaged,  and,  3)  the  servo  amplifier  package  caa  slipped 
on  its  shaft.  All  of  these  failures  occurred  with  the  art*  of  rotation 
iii  line  with  the  acceleration  force. 

Workmanship  discrepancies  led  to  a  number  of  failures.  For  example, 
(Figure  3)  during  vibration  testing  of  s  fuel  fill  and  drain  valve,  the 
position  switch  cover  assembly  came  loose*  Examination  of  the  assembly 
shoved  that  20  out  of  22  cover  screws  were  missing*  Inspection  was 
alerted  to  this  oversight  and  a  special  check  of  similar  valves  in  stock 
was  made* 

In  another  case,  a  pneumatic  relief  valve  va  examined  oy  x-ray  to 
investigate  the  reason  for  erratic  performan r  ,  It  was  noted  that  s 
spring  retainer  had  been  inserted  backwards,  permitting  the  spring  to 
drift  off  center.  (See  Figure  li) 

In  some  instances  a  series  of  failures  occurred  which  were  suffi¬ 
ciently  alike  to  fall  into  a  distinct  class.  One  case  of  s  typical  class 
of  failures  was  detected  when  several  coil  springs  mounted  in  rise-off 
and  staging  disconnect  valves  failed  during  the  test  program*  Analysis 
of  the  test  specimens  showed  that  the  coil  springs  would  bow  during  com¬ 
pression  and  while  operating  under  simulated  environmental  conditions 
would  rub  against  their  housing*  This  action  led  to  three  serious  weak¬ 
nesses)  1)  in  a  staging  disconnect  the  rubbing  action  of  the  coil  spring 
scraped  metal  fragments  off  the  housing,  causing  contamination, (See  Fig¬ 
ure  5)  in  addition  to  a  fractured  spring.  This  discrepancy  was  corrected 
by  Introducing  a  hard  metal  sleeve  in  the  housing  and  changing  the  coil 
spring  design;  2)  in  a  ground  rise-off  valve  the  bowed  spring  exerted  a 
aid/  load  on  its  housing,  causing  the  poppet  to  seize  in  an  open  posi¬ 
tion.  (See  Figure  6)  Such  a  failure,  if  occurring  during  actual  operat¬ 
ing  conditions,  would  have  resulted  in  loss  of  tank  pressurization.  This 
was  corrected  by  adding  a  spring  guide  and  a  hard  metal  sleeve.  Another 
example  of  a  bowed  spring  exerting  a  side  load  is  shown  in  the  x-ray 
photograph  (See  Figure  7),  of  a  large  fuel  staging  valve;  3)  the  spring 
on  a  hydraulic  relief  valve  (See  Figure  8),  bowed,  scoring  the  cylinder 
wall,  poppet  and  spring.  This  led  to  erratic  poppet  performance.  It 
was  corrected  by  introducing  a  telescoping  spring  guide  as  shown  in  the 
photograph.  The  failure  of  any  one  of  these  components  during  actual 
operating  conditions  would  have  precluded  the  fulfillment  of  the  intend- 
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ed  mission  —  a  high  price  to  pay  for  a  relatively  inexpensive  spring. 

Another  class  of  failures  was  attributed  to  "stress  raisers"  in  the 
fatigue  areas  of  the  specimens.  These  failures  usually  fell  into  the 
classification  of  insufficient  blend  radii  where  geometrical  discontinui¬ 
ties  exist  or  where  sharp  corners  act  as  focal  points  for  stress  concen¬ 
tration.  The  result  is  that  many  excellent  system  designs  are  negated 
by  a  critical  component  failure  that  may  have  been  avoided  by  merely 
haring  a  radius  instead  of  a  square  corner.  The  servo  valve  receptacle 
cap  shown  in  Figure  9  is  a  classic  example  of  how  a  fatigue  crack  not 
only  starts  at  a  siiarp  corner  but  often  will  propogate  from  one  fault  to 
another.  Figure  2  shows  a  staging  disconnect  in  which  there  was  no  pro¬ 
vision  for  a  relief  cut  at  the  end  of  the  threaded  portion;  the  sharp 
end  thread  became  the  focal  point  for  fatigue.  In  addition,  the  inner 
diameter  of  the  threaded  portion  was  severely  undercut.  In  Figure  10 
the  failure  of  a  telemetering  transducer  probe- illustrates  the  effect  of 
a  geometrical  discontinuity  without  a  blend  radius  with  a  comparatively 
large  sphere  at  the  end  of  the  slender  necked  down  area. 

Having  revealed  such  critical  weaknesses  .through  realistic  testing, 
the  Reliability  Organisation  was  able  to  institute  corrective  action. 
Further  studies  were  made  in  the  Reliability  Diagnostic  laboratory  using 
the  X-ray  fluoroscopy  facility  coupled  with  high  speed  motion  pictures. 
Designers,  manufacturers,  and  inspectors  were  alerted  to  the  critical 
conditions  that  exist  in  their  respective  areas.  This  is  done  most 
effectively  by  the  use  of  Reliability  Bulletins.  These  documents  are 
released  on  a  wide  distribution  basis  for  such  items  as  the  two  classes 
of  failures  discussed  above. 

C.  CONCLUSIONS 


Corrective  action  for  those  weaknesses  revealed  during  the  program, 
has  varied  from  intensified  inspection  techniques  and  improved  materials 
to  complete  redesign  or  change  in  vendors.  During  the  first  three  phases 
of  the  program,  an  average  of  6}%  of  the  component  types  tested  were  re¬ 
designed  to  correct  the  type  of. weakness  revealed  during  the  test.  The 
factors  responsible  for  detecting  such  a  large  number  of  weaknesses  and 
obtaining  subsequent  high  reliability  may  be  attributed  to  the  following* 

1.  Central  control  was  maintained  over  all 
testing  by  Reliability  as  an  independent 
organization. 

2.  Realistic  combined  environmental  condi¬ 
tions  were  applied. 
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3.  Tests  were  conducted  over  a  range  of 
environmental  intensities,  establishing 
safety  margins. 

U.  A  number  of  each  component  type  were 
tested  providing  performance  and  produc¬ 
tion  variation  data. 
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Carrying  out  thio  sane  general  testing  philosophy  since  the  early  phases 
of  the  program  ha a  enabled  Convair- Astronautics  to  realite"  the  continual 
improvement  of  the  atlas  Missile  components. 
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AFBMD  SPONSORED  COMBINED  ENVIRONMENTAL  TESTING  OP  FLUID  COMPONENTS 

Sherwin  Lewis 

Liquid  Rocket  Engines  Department,  Space  Technology  laboratories,  Inc. 

Ballistic  missiles  using  tanked  propellants,  which  require  pressuriza¬ 
tion,  pumping  and  controlled  flow,  need  valves,  regulators,  and  switches. 
Reliability  requirements  for  these  components  were  beyond  the  Btate  of 
the  art  at  the  outset  of  the  ICBM  development  programs.  Recognizing 
this,  the  Air  Force  Ballistic  Missile  Division  initiated  a  Valve 
Improvement  Program  under  the  technical  direction  of  Space  Technology 
laboratories,  Inc.  The  objectives  of  this  program  were,  first}  making 
a  tetter  and  more  complete  evaluation  of  existing  valve  and  switching 
hardware,  and  second;  to  provide  guidance  and  coordination  in  the 
development  of  new  components  which  would  meet  the  stringent  require¬ 
ments  of  ballistic  missiles. 

Primary  consideration  in  meeting  the  first  objective  of  better  testing 
and  evaluating  equipment  for  existing  valveB  was  directed  toward 
developing  techniques  and  methods  for  subjecting  these  components  to 
combined  environmental  conditions.  Some  attempt  had  been  made  at 
establishing  acceptance  and  qualification  tests  at  both  the  prime 
contractor  and  the  valve  vendors'  establishments  which  simulated  some 
of  the  characteristics  of  the  potential  environment.  But,  in  most  of 
these  cases,  the  urgency  inherent  in  the  delivery  schedules  resulted 
in  waivers  and  relaxed  environmental  demonstration  so  that  the  flow 
of  hardware  could  proceed.  These  methodB  produced  little  real  Infor¬ 
mation  as  to  the  location  of  the  basic  design  weaknesses  of  these 
components  and  contributed  relatively  little  to  the  Improvement  of  the 
product.  For  reasons  of  this  type,  the  AFBMD  program  was  established 
on  the  basis  that,  whenever  possible,  evaluation  of  valve,  regulator 
and  switch  components  would  be  made  under  full  flow  conditions  with 
actual  missile  fluids  and  subjected  to  a  combination  of  missile  environ¬ 
ments  including  vibration,  acceleration,  high  and  low  ambient  temperatures, 
altitude  and  humidity. 

To  fulfill  the  second  objective  of  the  program,  requests  for  bids  to 
develop  new  components  were  sent  out  to  the  valve  industry  and  seven 
contracts  resulted  from  approximately  100  proposals  received  and 
evaluated.  Part  of  the  effort  contracted  from  three  of  the  seven, 
Aerojet-General,  AlResearch,  and  the  MiBBlle  Division  of  North  American 
Aviation,  Inc.  was  to  undertake  the  development  of  combined  environ¬ 
mental  devices. 

The  first  combined  environmental  testing  in  the  AFBMD  program  was 
done  by  the  Propulsion  Laboratory  of  the  Wright  Air  Development  Center 
acting  Jointly  with  the  Aeronautical  Accessories  laboratory  of  that 
same  Center.  This  work  involved  the  use  of  a  large  centrifuge  table 
available  at  the  Aeronautical  Accessories  Laboratory.  Following  the 
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transfer  of  the  rocket  development  responsibilities  from  the  Wright 
Air  Development  Center  to  the  Air  Force  Flight  Test  Center  at  Edvards, 
California,  the  latter  has  developed  a  valve  testing  laboratory  which 
will  soon  be  complemented  by  a  combined  environmental  centrifuge 
capable  of  handling  the  largest  present  day  missile  fluid  system 
c_nponente.  In  addition  to  these  efforts,  two  commercial  laboratories 
also  have  Joined  the  effort  to  establish  suitable  combined  environmental 
apparatus . 

A  description  of  the  approaches  taken  by  each  of  these  organisations  vlll 
aid  in  an  understanding  of  the  accomplishments  already  achieved  and  the 
potential  which  remains  in  the  science  of  properly  evaluating  valve 
and  switch  components. 

At  the  Aeronautical  Accessories  laboratory,  a  Calidyne  Model  44 
shaker  with  a  force  output  of  four  hundred  fifty  (450)  pounds  was 
mounted  between  two  (2)  massive  steel  centrifuge  discs,  twelve  (12) 
feet  in  diameter.  (Figures  1,  2,  3)  The  large  mass  of  the  discs 
minimized  the  effects  of  vibration  reactive  forces.  The  vibrator  was 
modified  as  follows: 

Retaining  plates  were  made  to  support  the  shaker  casting,  wooden 
spacers  were  installed  to  support  the  field  winding,  additional 
flexures  were  fabricated,  and  a  positioning  indicator  and 
positioning  devices  were  devised. 

As  a  result  of  these  modifications,  the  output  of  the  vibrator  was  not 
adversely  affected  by  accelerations  as  high  as  forty-five  (45)  g.  The 
concepts  evolved  at  VADC  were  made  available  to  private  companies  for 
use  in  combined  environmental  apparatus.  Wright  Air  Development  Center 
conclusively  proved  that  combined  environmental  testing  was  possible 
e~d  practical. 

Aerojet-General  Corporation  chose  to  build  an  apparatus  in  which  the 
rotating  boom  was  mounted  on  a  6haker.  Aerojet  further  decided  to 
tackle  the  problem  of  designing  a  versatile  machine  to  control  the 
orientation  of  the  vectors  of  acceleration  and  vibration  so  that  vibra¬ 
tion  could  be  imposed  either  parallel  or  perpendicular  to  the 
acceleration.  The  major  portion  of  the  design,  effort  wa6  concentrated 
on  the  development  of  a  rocker  assembly  to  convert  the  vertical 
vibration  vector  to  a  horizontal  vector.  The  rotating  assembly  was 
secured  to  an  electrodynamic  shaker  by  means  of  a  pre-loaded  thrust 
bearing  assembly.  (Figures  4,  5,  6)  Work  by  this  contractor  was 
stopped  by  fund  limitations;  however,  at  the  time  of  suspension 
sufficient  progress  had  been  made  to  establish  the  feasibility  of 
the  approach.'  A  comparison  of  the  centrifuge-on- shaker  approach  with 
the  opposite  more  conventional  approach  of  placing  the  shaker  on  a 
centrifuge,  reveals  the  following  advantages  of  the  former: 
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1.  The  system  Is  more  compact .  Realistic  dimensions  would  have 
been  a  forty  (4o)  Inch  chamber  diameter  -and  forty  (4o)  Inches 
over-all  height  as  measured  from  the  shaker  table. 

2.  The  Installation  Is  cheaper.  If  a  ahoker  of  the  proper  size 
is  available.  This  advantage  1?  Largely  due  to  the  smaller 
size  of  the  system  and  to  the  simpler  plumbing  and  Instrumen¬ 
tation. 

3.  Slip-rings  are  not  required  to  supply  power  to  the  shaker. 

4.  The  shaker  is  readily  freed  for  other  tests. 

5.  Altitude  and  acceleration  transients  are  easily  produced  to 
simulate  missile  gyrations.  The  evacuating  equipment  can  be 
closely  coupled  and  the  centrifuge  portion  of  the  system  has 
low  inertia. 

6.  Fewer  slip-rings  and  swivel  connections  are  required  because 
the  heating,  cooling  and  evacuating  equipment  is  stationary. 

Conversely,  the  disadvantages  are: 

1.  Pronounced  acceleration  gradient  across  the  specimen  because 
of  the  relatively  small -centrifuge  radiuB.  Variations  are  as 
great  as  +  thirty  ( 30 )  per  cent  for  bulky  specimens. 

2.  The  system  is  limited  to  vibration  testing  of  specimens 
weighing  less  than  five  (5)  pounds. 

3.  The  ratio  of  total  vibrating  mass  to  specimen  mass  is  1  da¬ 
tively  large.  The  extraneous  masses  are  those  of  the  lower 
bearings,  lower  shaft  and  rocker  arms. 

4.  larger  vibrators  are  required. 

A  third  approach  was  taken  by  the  second  AFBMD- valve  contractor, 
AiResearch  Manufacturing  Division  of  The  Garrett  Corporation. 

AiResearch  also  decided  that  for  the  relatively  small  component  being 
developed,  and  with  the  availability  of  a  large  company  owned  shaker, 
the  optimum  configuration  was  centrifuge-on-shaker.  (Figure  7)  To 
keep  the  effort  within  finarcial  bounds,  AiResearch  decided  to  build 
a  device  limited  to  imposing  vibratory  excitation  perpendicn lar  to 
acceleration.  It  was  reasoned  that  this  would  be  useful  for  the 
development  testing  and  for  a  portion  of  the  qualification  testing 
and  that  eventually  commercial  laboratories  would  supply  the  remaining 
vector  relationships .  The  device  built  by  AiResearch  works  as  follows: 

Sinusoidal  vibration  with  frequencies  up  to  two  thousand  (200C) 
cps,  applied  by  the  shaker,  is  transmitted  to  a  cart.  The  cart 
and  counterweight  travel  in  a;i  annular  track  at  the  end  of  a 
ten  (lO)  inch  radius  arm  centrifuge  on  the  top  of  the  shaker 
head.  (Figure  8)  The  valve  is  mounted  on  this  cart.  There  are 
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ball  bearings  beneath  the  center  of  the  cart  and  a  set  of  out¬ 
board  bearings  on  top  of  each  end  of  the  cart.  The  center 
bearings  run  on  the  lower  track  surface  while  the  outboard 
bearings  run  on  the  upper  track  surface.  The  vibratory  forces 
a»*e  transmitted  from  these  surfaces.  The  bearings  are  internally 
pre-loaded  with  a  force  greater  than  the  vibratory  forces  to 
prevent  brlnnellng.  The  shaker  is  required  to  shake  the  track, 
cart,  the  unit  being  tested,  the  counterweight,  and  the  insulated 
pressure  vessel  which  surrounds  the  teBt  article.  A  hollow  shaft 
driven  by  an  electric  motor  drives  the  radius  arm  from  above  and 
serves  as  a  conduit  for  the  thirty  (30)  channel  leads  and  plumbing 
for  the  vacuum  system.  High  pressure  helium,  as  well  as  the 
regulator  and  heat  exchanger  for  conditioning  the  fluid,  is 
stored  on  the  rotating  components  prior  to  teBt. 

The  third  AFHMD  contractor  undertaking  the  development  of  a  combined 
environmental  apparatus  was  the  Missile  Division  of  North  American 
Aviation,  Inc.  Like  the  other  two  contractors,  NAA  had  a  total  fixture 
weight  of  less  than  five  (5)  pounds  and  desired  to  utilize  available 
equipment  without  impairing  individual  capabilities]  however,  the 
end  result  was  strikingly  dissimilar.  The  NAA  decision  was  to  place  a 
conventional  M-3  C6C  shaker  with  a  rated  force  output  of. two  hundred 
(200)  pounds  and  an  upper  frequency  limit  of  five  thousand  (5000)  cps 
on  a  Genesco  Model  C-159,  ninety-two  (92)  inch  diameter  centrifuge  with 
a  rated  capacity  of  two  thousand  (2000)  g  pouodB  and  a  maximum  static 
load  capacity  of  one  hundred  (100)  pounds.  (Figure  9)  Ab  a  result  of 
analytical  studies  and  static  tests,  the  centrifuge  beam  was  stiffened. 
Two  vibration  exciters  with  a  properly  designed  whiffletree  were  used 
to  counteract  loads  on  the  centrifuge  bearings  making  possible  the  use 
of  weights  more  than  eight  (8)  times  the  rated  capacity  of  the  accelera¬ 
tor.  Difficulties  with  swivels  were  worked  out;  a  special  air  cylinder 
to  locate  the  moving  element  in  the  centrifugal  force  was  devised;  and 
6lip-rings  were  debugged.  High  and  low  temperatures  were  obtained  on 
the  centrifuge  by  impinging  gas  on  the  test  valve.'  For  gross  loads  on 
the  6haker  of  about  eight  (8)  pounds  this  device  works  very  satisfac¬ 
torily.  Because  of  the  nature  of  the  component,  simultaneous  vacuum 
was  not  required. 

To  provide  additional  test  and  evaluation  capability  in  addition  to  the 
in-house  capability  of  WADC  and  AFFTC,  the  Air  Force,  early  in  1959> 
Invited  bids  to  procure  environmental  test  services.  Five  proposalo 
were  received  from  test  laboratories  and  two  laboratories,  Btellardyne 
Laboratories  and  Wyle  Laboratories,  vere  chosen  by  the  Air  Force 
Ballistic  MlsGile  Division  to  evaluate  current  components  under  a  call 
contract  with  technical  direction  from  Space  Technology  Laboratories,  Inc. 

Stellardyne  Laboratories  has  built. a  combined  environmental  apparatus 
consisting  of  a  ten  (10)  foot  centrifuge  arm  fabricated  from  steel 
tubing  with  a  Savage  V1000  vibrator  especially  adapted  to  centrifuge 
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applications  by  a  unique  method  of  alignment  retention  of  the  moving 
element.  (Figure  10)  Stellardyne  also  has  had  to  work  out  the  varied 
problems  of  compensating  the  weight  of  the  moving  element  during  radial 
vibration,  slip-ring  selection,  rotary  Joints,  imposing  ten5>erature, 
and  recording  instrumentation.  Stellardyne  is"  now  currently  testing 
ballistic  missile  components  to  Air  Force  requirements. 

Probably  the  most  complete  combined  environmental  apparatus  in  the 
AF3MD  program  was  built  by  Wyle  Laboratories.  (Figure  ll)  This 
Incorporates  a  nine  (9)  foot  radius  centrifuge  mounting,  carrying  an 
M-B  C25H  vibration  exciter  with  sinusoidal  and  random  capability.  The 
centrifuge  arm  consists  of  two  (2),  sixteen  (l6)  inch  I-beams  with 
suitable  structural  ties,  mounted  on  a  hollow  6haft  through  tapered 
roller  bearings.  The  combined  environmental  system  provides  simul¬ 
taneously: 


A. 

Vibration  to  35*00  pounds  force,  20(0 

cps . 

B. 

Accelerator  to  20  g. 

C. 

Altitude  programmed  to  250,000  feet. 

D. 

Temperature  programmed  from  -300°F  to 

1  +500°F. 

E. 

Humidity  to  100  per  cent. 

F. 

Liquid  Oxygen,  gaseous  oxygen,  fuel, 
helium  flow  as  required. 

hydraulic  fluid,  and 

The  machine  incorporates  the  following  innovations: 

1.  Tension  rods  from  the  shaker  trunnlonB  to  the  center  member 
dynamically  Isolate  the  shaker  from  the  beam  horizontally, 
and  a  cantilevered  leaf  spring  takes  the  vertical  load. 

2.  A  special  auxiliary  flexure  was  devised  to  maintain  coll 
centering  in  the  M-B  C25H  shaker. 

3.  A  special  environmental  chamber  was  devised  to  fit  on  the 
shaker;  this  chamber  Incorporates  an  air  spring  to  provide 
centering  of  the  shaker  table  during  horizontal  vibration 
conditions . 

4.  The  sllp-r.'ag  assembly  includes  twenty  (20)  shielded  instru¬ 
mentation  rings  and  fifty  (50)  rings  rated  from  ten  (10)  to 
one  hundred  (100)  amperes. 

5-  The  system  includes  a  concentric  tubing  swivel  assembly.  A 
vacuum  can  be  maintained  in  the  four  (4)  inch  outer  swivel 
tube  while  cryogenic  fluid  is  carried  by  the  one  (l)  inch 
inner  swivel  tube. 

6.  In  conjunction  v:.th  this  system,  the  helium  system  can  be 
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programed  as  to  temperature  (-150°F  to  +500°F  and  flow 
rate  (0  to  2  nps)  and  altitude  can  be  programmed. 

Th  •  combined  environmental  apparatus  of  the  AlReaeurch  Division  of 
The  G',,rett  Corporation  has  not  yet  made  Its  debut;  and  as  noted 
above,  the  efforts  of  Aerojet-Oeneral  were  dropped  because  of  finan¬ 
cial  limitations.  The  early  VADC  effort  was  primarily  directed  to 
wording  out  difficulties  of  the  combined  environmental  apparatus 
proper.  Although  combined  environmental  testing  of  propulsion  system 
solenoid  ‘valves  was  accomplished  and  interesting  behavior  was  noted, 
it  was  not  established  that  the  combined  environment  of  acceleration, 
vibration  and  temperature  was  more  significant  than  the  simple  combi¬ 
nation  of  vibration  and  temperature. 

Results  obtained  to  date  have  come  mainly  from  the  efforts  of  NAA, 
S+ellardyne,  and  Wyle  laboratories. 

The  programs  of  combined  environmental  testing  are  all  in  their  early 
stages,  and  the  results  do  not  yet  show  a  definite  pattern.  Some 
examples  of  the  varied  results  are  given  below: 

At  one  extreme  of  the  Bpectrum  of  testing  results  is  a  missile 
propellant  tank  pressure  regulator.  The  valve  went  completely 
unstable  during  the  test  and  finally  ceased  to  regulate.  Subse¬ 
quent  Inspection  showed  that  the  poppet  valve  In  the  6hut-off 
assembly  had  worn  severely  and  was  the  primary  cause  of  the 
malfunction.  The  specimen  had  beeg  subjected  to  a  preconditioning 
humidity  and  icing  cycle  at  14  +  4  F.  The  specimen  was  then  sub¬ 
jected  to  a  centrifugal  acceleration  of  ten  (10)  g  applied  in  the 
Z  axis,  stfid  random  vibration  at  a  level  of  ninetenn  (19)  g  RM3  vus 
applied  in  the  Y  axis.  Once  these  conditions  were  established,  a, 
temperature-altitude-flow  program  simulated  fissile  conditions  b^ 
varying  the  inlet  flow  temperature  from  +100  F  to  -100  F  to  +4-50  F 
along  a  specified  curve,  and  the  ambient  pressure  was  varied  from 
14.7  psia  to  ballistic  missile  altitude.  This  was  a  very  severe 
test  and  the  regulator  failed.  Needless  to  say  it  is  desirable 
that  ballistic  missile  regulators  pass  such  tests,  but  at  the 
present  time  the  ballls-ic  missile  program  cannot  afford  to 
eliminate  regulators  that  do  not  pass  such  a  test.  In  discussing 
this  regulator  the  prime  contractor  project  engineer  stated  that 
laboratory  test  results  are  considered  only  an  Index  of  comparison 
and  that  flight  Is  the  only  valid  basis  of  evaluation. 

At  the  other  extreme  of  the  test  result  spectrum,  Stellardyne 
Laboratories  tested  a  ballistic  missile  accumulator  relief  valve 
and  a  low  flow  engine  regulator  under  conditions  of  eight  (8)  g 
acceleration  and  twenty-five  (25)  g  vibration.  Generally,  there 
was  no  apparent  effect. 


Between  the  extremes  of  "accelerated  failure"  and  "no  apparent  effect", 
there  is  an  area  that  requires  close  scrutiny. 

•'orcn  American  Aviation,  Inc.,  Missile  Division  conducted  combined  envi¬ 
ronmental  testing  during  the  development  of  the  solenoid  valve  in  the 
AFBMD  New  Valve  Design  and  Development  Program.  During  this  testing  it 
was  discovered  that  under  conditions  of  acceleration  in  line  with  the 
solenoid  slug  and  vibration  perpendicular  to  the  acceleration,  the 
solenoid  valve  leaked  excessively.  Single  environmental  vibration 
testing  at  high  g  levels  produced  a  similar  effect.  However,  it  was 
necessary  to  impose  an  artificially  high  vibration  level  that  bad  no 
correlation  to  missile  flight  conditions.  Combined  environmental 
testing  has  direct  correlation. 
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Another  malfunction  occurred  during  the  qualification  testing  of  the 
Freb&nk  Company  pressure  switch,  another  component  deveioped  in  the  AFBMD 
Hew  Valve  Design  and  Development  Program.  In  the  initial  phases  of  the 
qualification  testing,  these  pressure  switches  were  subjected  to  dual 
environments  of  vibration  and  temperature.  The  pressure  switches  were 
Subjected  to  twenty-five  (25)  g  vibraJLion  scans  from  twenty  (20)  to  two 
thousand  (2000)  cycles  per  second  while  immersed  in  temperature  control 
boxes  at  -300  F  and  +  300  F.  These  components  showed  no  tendency  to 
chatter  or  malfunction.  However,  during  combined  environmental  testing 
a  switch  failed.  Investigations  showed  that  the  failure  was  the  result 
of  a  fabrication  probLem  in  which  the  high  heat  of  soldering  had  caused 
the  electrical  elements  to  lose  their  temper  and  distort  the  plastic 
case.  The  intensive  testing  of  scrambled  temperature,  attitude  and 
vector  combinations  rapidly  brought  about  a  failure  that  would  have  been 
attributed  to  random  causes  on  a  missile. 


Results  of  combined  environmental  testing  are  only  beginning  to  become 
available  from  the  AFBMD  programs. 

Combined  environmental  testing  Is  currently  proceeding  in  both  the  New 
Valve  Design  and  Development  Program  and  in  the  evaluation  of  current 
ballistic  missile  hardware.  Each  test  adds  another  bit  of  data  for  the 
evaluation  of  this  mode  of  testing  for  increasing  the  reliability  of 
ballistic  missile  component;.  In  the  meantime,  consider  three  bits  of 
data;  one,  a  complex  ballistic  missile  propellant  tank  regulator  that 
failed  under  combined  environmental  testing;  two,  the  failure  of  a 
pressure  switch  tha  .  had  completed  intensive  single  environmental 
testing  to  such  a  d< gree  that  all  engineers  associated  with  it  were 
positive  that  it  would  pass;  and  three,  a  prime  contractor  that  con¬ 
siders  laboratory  to.  ting  insufficient  evidence  to  determine  flight 
worthiness.  Each  instance  indicates  present  testing  methods  are 
insufficient . 
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The  significance  of  combined  environmental  testing  is  that  it  will 
Judge  a  component  by  a  much  more  rigorous  standard  than  the 
wisest  of  engineers  uses  when  evaluating  the  results  of  single 
environmental  testing.  If  the  design  meets  this  rigorous  test, 
the  prime  contractor  can  have  a  new  confidence  in  the  inherent 
reliability  of  the  component. 
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RELIABILITY  THROUGH  ENVIRONMENTAL  TESTS 


I 

T.  L  Commons 
Bendix  Mishawaka  Division 
Mishawaka,  Indiana 


INTRODUCTION 

Environmental  testing  today  Is  done  under  a  wide  range  of  programs,  given 
various  names  and  classifications  an"!  performed  In  varying  degrees  of  sever¬ 
ity.  At  Bendlx-Mlsslles,  we  have  four  categories  or  .classifications  of  tests  for 
evaluating  our  product:  Design  Margin  Evaluation,  Design  Evaluation,  Qiallty 
Assurance,  and  Production  Test.  There  aie,  undoubtedly,  numerous  other 
classifications  and  names  given  by  other  manufacturers  for  environmental  tests'- 
which  may  or  may  not  have  the  same  definition  as  the  tests  performed  at  Bendlx- 
Mlsslles.  However,  all  the  names,  classifications,  and  severity  of  tests  have 
only  one  ultimate  purpose— reliability. 

The  Engineering  Department  at  Bendix  has  an  evaluation  laboratory  which  eval¬ 
uates  and  qualifies  small  piece  parts  such  as  capacitors,  resistors,  etc. ,  but 
this  paper  deals  only  with  those  tests  which  are  performed  by  the  Quality  and 
Reliability  Department,  henceforth  called  simply  "Quality"  Department,  either 
alone  or  In  conjunction  with  the  Engineering  Department.  This  doeB  not  mean 
that  Quality  Is  out  of  the  picture  In  regards  to  piece  parts.  On  the  contrary, 
Quality  Is  very  much  concerned  with  the  parts  evaluated  and  the  results  obtained. 
However,  test  procedures  are  written,  tests  performed,  results  analyzed,  and 
conclusions  drawn  entirely  by  the  evaluation  lab.  In  the  case  of  the  other  tests 
performed  on  the  larger  Items,  such  as  gyros.  Inverters,  modules,  systems 
and  missiles.  Quality  and  Engineering  together  determine  what  tests  are  to  be 
performed  and  the  Quality  Department  is  then  responsible  for  writing  the  test 
plans  and  test  procedures,  performing  the  tests,  and  writing  the  reports.  Dur¬ 
ing  the  entire  testing  phase,  numerous  conferences  are  held  with  Engineering  to 
clear  up  problems  that  arise,  or  to  change  the  test  plans  as  a  result  of  some 
previous  test.  Figure  1  is  a  block  diagram  of  the  Quality  organization  within 
Bendlx-Mlsslles. 

I  might  explain  the  difference  between  "piece  part,"  "component,"  and  "sub- 
assembly"  as  they  are  used  by  Bendix.  The  major  difference  between  piece  part 
and  component  Is  the  dollar  value.  The  term  "piece  part"  Is  applied  to  all  Items 
costing  less  than  an  arbitrary  figure  of  $50.  00.  A  "component,"  besides  being 
expensive,  may  also  be  considered  as  consisting  of  a  group  of  piece  parts  made 
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up  into  an  assembled  or  a  potted  unit.  A  ''sub-assembly"  usually  refers  to  a 
mechanical  sub-system  on  the  same  level  of  complexity  as  a  component.  An 
example  of  each  of  these  would  be  as  follows:  piece  parts  -  resistors,  capaci¬ 
tors,  tubes;  components  -  magnetrons,  klystrons,  Inverters,  gyros;  sub- 
assemblies  -  fuel  pump,  hydraulic  pump,  actuatora,  serves. 

RELIABILITY  EFFORT 

Reliability  is  an  essential  organization  at  Bendlx-Misstles  and  their  activities 
are  varied.  The  Reliability  Oroup  at  Bendtx  Is  (n  the  Quality  Department  and 
reports  directly  to  the  Quality  Manager  as  Indicated  In  Figure  1.  Reliability  Is 
responsible  for  an  extensive  failure  reporting  system  which  covers  the  sub¬ 
contractor,  in-plant,  and  field  operations.  The  group  also  provides  engineering 
Investigation  and  follow-up,  statistical  and  mathematical  assistance  In  setting 
up  and  analyzing  tests  and  test  results,  statistical  and  mathematical  analysis  of 
failure  data,  and,  also,  conducts  quality  examinations  and  Quality  Assurance 
Tests. 

As  part  of  the  engineering  follow-up,  Individuals  In  the  group  are  responsible 
for  monitoring  various  activities  within  the  plant.  One  of  these  activities  1b  the 
Environmental  Test  Laboratory  where  all  DeBtgn  Margin  Evaluation  (DME),  De¬ 
sign  Evaluation  Tests  (DET),  and  Quality  Assurance  Test  (QAT)  Programs  are 
conducted.  On  all  test  programs,  we  In  Reliability  provide  Engineering  with 
historical  failure  data  as  well  as  engineering  and  mathematical  assistance  in 
analyzing  failures  that  have  occurred  under  test.  We  also  assist  In  evaluating 
the  changes  that  have  been  Initiated  as  a  result  of  the  failures.  An  example  of 
this  type  of  assistance  occurred  during  a  DET  program.  Two  crystal  diodes 
had  failed  during  test.  One  of  the  test  engineers  on  the  program  recalled  that 
they  had  had  some  failures  on  these  diodes  on  the  production  floor.  A  request 
was  made  to  Reliability  to  Investigate  the  failure  history  of  this  diode.  The  In¬ 
vestigation  proved  that  the  diode  had  a  failure  history  at  one  time  but  that  steps 
had  already  been  taken  by  Reliability  to  cure  this  problem,  which  was  due  to 
mishandling  by  assembly  and  test  personnel.  Later  reports  indicated  that  the 
failure  rate  had  definitely  declined. 

Another  Instance  occurred  when  a  plug  shorted  out.  One  of  the  follow-up  engi¬ 
neers  remembered  that  a  similar  failure  had  occurred  a  couple  of  months 
previously.  It  was  assumed  by  the  test  personnel  at  the  time  that  the  plug  was 
mls-wlred  since  the  short  existed  between  a  live  pin  and  a  spare  pin  and  the 
plug  was  discarded.  However,  the  junction  on  the  unit  to  which  the  spare  pin 
Is  mated  Is  wired  to  ground  for  other  test  purposes.  When  the  second  failure 
occurred,  both  plugs  were  depotted  by  Reliability  and  It  was  discovered  that  the 
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plugs  were  wired  with  bare  wires  and  that  one  of  the  wires  had  shorted  against 
a  spare  pin.  It  was  also  discovered  that  the  automatic  test  equipment  used  for 
testing  the  plugs  had  not  been  wired  to  detect  shorts  that  existed  betreen  used 
aud  unused  pins.  Corrective  measures  were  taken  Immediately. 

The  failures  mentioned  here  were  not  peculiar  to  the  environmental  test  being 
performed  since  similar  failures  occurred  elsewhere.  But  these  two  examples  . 
do  point  out  some  of  the  assistance  provided  by  the  Reliability  Group.  Some  of 
the  failures  that  were  detected  as  a  result  of  the  test  program  will  be  shown 
later  along  with  some  of  the  test  setups  and  environmental  equipment  used. 

Reliability  has  also  provided  extensive  assistance  in  evaluating  the  accuracy, 
stability,  and  reliability  of  some  automatic  test  equipment.  The  analysis  sec¬ 
tion  of  the  Reliability  Group  devised  tne  test  sequence  and  specified  the  data  to 
be  recorded.  The  data  was  analyzed,  results  derived,  and  recommendations 
made  regarding  maintenance  and  adjustment  schedules.  A  complete  description 
of  this  test  Is  given  In  another  paper.  *  Another  test  statistically  controlled  by 
Reliability  was  an  interchangeability  test.  This  was  performed  by  exchanging 
parts  and  units  from  several  different  missiles  in  a  statistically  controlled  se¬ 
quence  and  determining  the  effects  on  missile  performance. 

TEST  PROGRAM 

At  this  time,  I  would  like  to  discuss  the  philosophy  and  purpose  which  Bendlx- 
MlsBlles  has  In  performing  these  various  tests.  Figure  2  Indicates  the  'ideal" 
sequence  which  we  attempt  to  follow  In  carrying  out  this  entire  test  program. 

As  Illustrated  In  this  diagram,  Design  Margin  Evaluation  Tests  (DME)  are 
usually  performed  on  one  or  more  of  the  first  engineering  models  produced. 

The  environments  to  which  these  units  are  then  subjected  determine  the  ultimate 
capabilities  of  the  missile  design.  The  tests  as  performed  are  necessarily 
detrimental  to  the  test  specimens  since  to  determine  the  ultimate  capabilities 
means  Invariably  testing  to  destruction.  However,  we  feel  that  this  Is  a  very 
necessary  expenditure  and  a  profitable  venture  In  the  long  run.  The  changes 
necessary  to  correct  flaws  of  design  or  construction  discovered  during  these 
teats  are  put  Into  the  pre-production  or  production  prototype  modsls. 

Aa  Indicated  In  Figure  2,  it  is  at  this  stage  of  the  test  program  that  the  Design 
Evaluation  Tests  (DET)  are  b;gun.  Although  the  same  teste  are  conducted  In 
both  DET  and  DME,  the  severity  level  of  the  environment  Is  considerably  de¬ 
creased  In  the  DET.  The  pur  r.se  of  DET  Is  to  determine  If  the  missile,  mod¬ 
ule  or  component  will  meet  the  requirements  as  called  out  In  the  design  specifi¬ 
cations.  This  means  testing  t>  the  limit  of  the  specification  requirement  and 
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no  further,  and  also  tests  which  are  not  necessarily  destructive  In  nature.  After 
a  sufficient  amount  of  rework,  the  test  items  are  frequently  used  for  special 
engineering  flight  tests  or  additional  marginal  testing.  If  any  design  changes 
are  deemed  necessary  as  a  result  of  the  DET  program,  these  changes  are  then 
put  into  the  early  production  models. 

The  next  phase  of  the  test  program  is  a  joint  phlse  with  both  Quality  Assurance 
Tests  (QAT)  and  Production  Tests  being  performed  at  the  same  time.  Produc - 
t'on  Test  is  the  normal  routine  method  of  providing  the  customer  with  the  assur¬ 
ance  that  he  is  receiving  a  product  that  ts  functionally  correct.  Production 
Testing  consists  of  operating  the  units  under  static  environmental  conditions  by 
applying  electrical  and  mechanical  power  and  varying  the  input  parameters  in  a 
predetermined  sequence  to  ascertain  If  the  unit  responds  correctly.  Other  than 
few  words  just  mentioned,  I  do  not  Intend  to  spend  any  more  time  on  Produc¬ 
tion  TeBt,  but  will  go  on  to  the  final  subject  as  Indicated  in  Figure  2,  Quality 
Assurance  Testing.  Where  Production  Tests  are  simplified  acceptance  tests 
based  upon  minimum  acceptance  criteria,  QAT  are  tests  taken  directly  from  the 
DET  test  program,  and  DET  test  procedures  are  used.  Production  Tests  are 
performed  on  every  major  unit  and  every  missile  as  well  as  some  sub-assem¬ 
blies,  components  and  parts  while  QAT  is  performed  on  a  sample  basis  on  these 
items.  _ 

There  is  one  major  difference  between  DET  and  QAT*  Each  missile  and  each 
module  which  1b  used  for  DET  receives  the  entire  sequence  of  tests,  hi  other 
words,  each  unit  is  tested  to  the  entire  design  specification.  In  QAT,  each 
sample  of  missiles  and  modules  is  subjected  to  only  a  few  of  the  tests  at  a  time, 
but  over  a  series  of  several  samples,  all  of  the  tests  performed  In  DET  will 
have  been  performed  under  QAT. 

On  the  component  and  sub-assembly  level,  each  sample  in  QAT  will  receive 
only  those  tests  which  are  judged  to  be  the  most  critical  to  the  construction  and 
design  of  these  parts,  and  these  tests  will  remain  the  same  on  all  samples  un¬ 
less  a  design  change  reflects  a  change  in  the  sensitivity  of  the  unit  to  the  envi¬ 
ronment  to  which  it  is  being  subjected.  Piece  parts  are  in  a  slightly  different 
situation  from  the  categories  of  units  just  presented  in  that  each  lot  of  piece 
parts  received  is  sampled  and  each  sample  is  tested  to  the  environments  Judged 
to  be  the  most  critical  for  that  type  of  part. 

Reliability  In  conjunction  with  the  Test  Engineering  Oroup  conducts  the  entire 
QAT  program.  Test  Engineering  Is  responsible  for  the  larger  Items  such  as 
missiles,  modules,  assemblies  and  systems,  with  Reliability  providing  mathe¬ 
matical  and  engineering  follow-up  assistance.  Reliability  Is  responsible  for 
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conducting  the  test  program  on  small  piece  parte,  components,  and  sub-assem¬ 
blies  with  the  assistance  of  Test  Engineering  and  Evaluation  Engineering. 

To  illustrate  In  more  detail  how  the  QAT  program  operates  and  to  what  extent 
the  tests  are  performed.  Figure  3  gives  what  might  be  considered  an  operating 
schedule  which  could  be  used  on  six  (6)  items.  These  Items  could  be  missiles, 
modules,  or  assemblies  and  would  not  necessarily  be  limited  to  six.  The  num¬ 
ber  of  items  tested  would  depend  on  two  things;  the  capabilities  of  your  testing 
laboratory  unless  you  contracted  your  testing  out  to  someone  else,  and  the  num¬ 
ber  of  Items  you  are  producing  over  a  given  Interval. 

You  will  notice  that  Figure  3  Hats  only  12  codes.  There  could  be  many  more  In 
which  case  you  would  change  your  testing  schedule  accordingly.  Code  number 
12  '.s  used  to  provide  sufficient  time  and  funds  In  your  program  for  any  possible 
rework  that  may  be  necessary  as  a  result  of  the  tests  performed  and  for  the 
necessary  Production  Tests  after  rework.  Because  QAT  ts  not  designed  to  be 
detrimental  and  the  number  of  tests  performed  on  any  one  Item  Is  limited,  the 
units  are  normally  considered  acceptable  by  the  customer  after  production  re¬ 
test  operation.  In  this  instance,  the  number  of  tests  being  performed  on  any 
one  unit  has  been  limited  to  four. 

There  is  no  real  need  to  adhere  to  a  firm  schedule  or  test  sequence  such  as 
presented  above.  Occasions  may  arise  when  a  unit  type  develops  a  series  of 
troubles  during  the  program  and  a  close  periodic  check  Is  made  over  a  period 
of  time  where  the  same  test  Is  repeated  on  succeeding  QAT  units  until  you  are 
sure  that  the  trouble  has  been  adequately  corrected.  Also,  the  customer  may 
have  some  complaints  from  the  operations  out  In  the  field,  or  In  our  case  the 
fleet,  and  the  customer  desires  that  you  prove  out  some  units  In  the  QAT  pro¬ 
gram  under  the  conditions  which  have  shown  up  a  problem  area. 

As  mentioned  previously,  the  QAT  program  conducted  on  missiles,  modules, 
and  other  major  Items  Is  performed  under  the  cognizance  of  the  Test  Engineer¬ 
ing  Group.  This  group  Is  responsible  for  writing  the  test  plans  and  test  pro¬ 
cedures  and  for  making  known  to  Engineering  and  Quality  any  .‘allures  that  de¬ 
velop  as  a  result  of  the  tes.s  conducted.  To  Inform  the  various  organizations  of 
these  failures,  two  forms  are  used.  (1)  A  TALOS  Discrepancy  Report  known  as 
a  Test  Tag,  Form  494.  This  Is  the  normal  discrepancy  reporting  test  tag  used 
throughout  the  plant  and  Is  the  reporting  form  for  which  the  Reliability  Group  Is 
responsible.  (2)  A  two  page  report  form  referred  to  as  a  Corrective  Action 
Report  (CAR).  Figure  6  shows  the  Test  Tag,  Form  494,  and  Figures  7  and  8 
show  the  two-page  document  used  by  Test  Engineering. 
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Reliability  Immediately  receives  a  copy  of  page  1  of  the  CAR  as  well  as  the  Test 
T.  g.  If  the  failure  Is  such  that  comments  and  analysis  axe  required  from  Engi¬ 
neering,  Engineering  then  fills  out  page  2  and  returns  It  to  the  cogntxant  test 
engineer  with  a  copy  also  being  sent  to  the  Reliability  Group.  From  past  fail¬ 
ure  tL.',a  and  engineering  knowledge.  Reliability  determines  If  the  recommended 
action  Is  sufficient  and  If  the  comments  made  are  appropriate.  When  the  tests 
are  completed,  a  complete  report  Is  made  to  the  customer,  Indicating  the  test 
performed,  discrepancies  found  and  action  taken.  When  all  the  action  to  be 
taken  Is  completed,  a  supplemental  report  is  Issued  to  the  customer  and  the 
entire  cycle  starts  over  again. 

In  regards  to  the  QAT  program  being  conducted  at  the  present  time  by  tho  Relia¬ 
bility  Group  on  piece  parts,  components,  and  sub-assemblies,  this  operation  Is 
performed  both  at  Bendlx-Mlsr'les  and  at  our  suocontrrctors.  Figure  4  de¬ 
scribes  this  program.  The  Buocontr&ctors  are  responsible  for  about  90%  of  the 
testing  effort  on  the  piece  part  program  and  the  Mtcstle  Plant  accounts  for  the 
other  10%  .  On  the  other  hand,  Bendlx-Misslles  conducts  and  performs  all  the 
testing  on  the  components  and  sub-assembly  QAT  program.  Figure  4  Is  Just  an 
Indication  of  the  over-all  flow  of  piece  parts  and  test  results  and  Is  given  to 
show  the  participation  by  the  various  organisations  In  the  test  program.  Fig¬ 
ure  S  goes  Into  more  detail  on  the  present  QAT  operation  on  piece  parts  which 
wae  my  main  interest  at  the  time  this  paper  was  written. 

QAT  Is  a  relatively  recent  Idea  at  Bendlx-Misslles  and  was  Inaugurated  only  a 
little  more  than  a  year  ago.  As  In  most  cases,  we  started  small  and  planned  to 
build  up  to  a  full  scale  QAT  program  after  we  had  learned  more  about  what  we 
were  Intending  to  do  and  to  accomplish.  For  this  reason,  you  might  say  we  have 
had  three  different  QAT  programs  on  piece  parts.  For  discussion  purposes  let 
us  refer  to  them  as  the  "Introductory,  "  "Intermediate,  "  and  "standard. "  This  Is 
not  to  preclude  further  modifications  of  the  approach  used,  but  simply  to  define 
the  programs  which  have  been  considered  and  used  to  date  (reference  Figure  5). 

The  "Introductory  '  program  was  Intended  primarily  as  a  learning  process  for 
the  Reliability  Group  Planning  ind  Te9t  engineers.  In  this  program  we  tried  to 
establish  our  ground  rules,  our  objectives  and  our  methods.  Engineering  judg¬ 
ment  was  tested  and  refined.  Specifications  were  reviewed,  test  methods  studied, 
and  various  blind  alleys  and  impracllcalltles  rejected.  All  testing  was  performed 
at  Bendlx-Misslles  using  the  facllttles  of  the  Environmental  Test  Lab.  Although 
a  tentative  test  plan  was  prepared  and  approved  before  testing  started,  detailed 
test  plans  and  procedures  were  not  prepared  In  advance.  A  Reliability  Engineer 
assigned  to  the  project  coordinated  all  testing,  procured  parts  and  equipment, 
monitored  operations,  and  made  decisions  on  the  spot.  The  logs  and  test  reports 
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prepared  for  this  program  formed  the  basis  for  further  study  and  development. 
Although  this  was  a  low  budget  program  and  sample  sizes  were  comparatively 
small,  results  were  useful  In  taking  action  on  poor  quality  hardware.  Fortu¬ 
nately,  from  the  QAT  viewpoint,  about  midway  through  this  program  a  serious 
part  quality  problem  was  found  In  production  testing.  This  problem  would,  In 
all  likelihood,  have  been  detected  and  eliminated  by  a  complete  QAT  program  if 
one  had  been  in  operation  at  the  time.  The  savings  In  cost  and  time  which  QAT 
could  have  effected  were  quite  helpful  In  showing  top  management  the  need  for 
QAT. 

As  soon  as  possible  after  the  "Introductory"  program  got  underway,  preliminary 
planning  for  the  "Intermediate"  program  was  started.  Time  and  funds  available 
were  again  somewhat  limited,  although  considerably  greater  than  for  the  Intro¬ 
ductory  program.  A  "General  Test  Plan  for  QAT  Program"  was  prepared  and 
approved  by  the  Navy.  This  plan  established  the  basic  concept  and  approach  to 
be  followed.  A  second  'Test  Plans  for  Piece  Parts  QAT"  was  then  prepared  to 
define  In  some  detail  the  tests  to  be  performed  on  various  classes  and  types  of 
parts,  components,  and  sub-assemblies.  Test  Requirement  Sheets  were  In¬ 
cluded  for  each  type  of  Item;  these  sheets  defined  the  environments  and  operating 
conditions  to  be  imposed  during  test.  As  a  rough  rule  of  thumb,  we  attempted 
to  restrict  the  tests  to  the  four  to  six  most  critical  tests  In  the  DET  sequence. 

In  all  cases,  loads  and  environments  were  held  to  the  level  Imposed  by  the  pro¬ 
curement  specification.  These  sheets  were  prepared  by  Reliability  Engineers 
assigned  to  the  project,  and  were  discussed  and  coordinated  with  the  Test  Engi¬ 
neering  and  Evaluation  Engineering  Departments. 

Testing  was  conducted  both  at  Bendlx-Mlsslles  In  the  Envlronu  ,  est  Lab, 

and  at  the  plants  of  major  subcontractors  In  the  program.  Special  purchase 
orders  were  written  to  authorize  and  fund  the  subcontractor  effort.  Each  sub¬ 
contractor  was  required  to  determine  a  sequence  of  testing  parts  to  ensure  the 
most  effective  use  of  funds  and  to  minimize  the  amount  of  production  rework 
necessitated  by  the  discovery  of  bad  qual'ly.  In  other  words,  each  subcontractor 
was  asked  to  review  his  own  records  and  exercise  his  own  Judgment  to  ensure 
testing  the  most  critical  It'  ms  first.  This  scnedule  and  the  detailed  test  pro¬ 
cedures  written  by  the  various  subcontractors  were  reviewed  and  approved  by 
the  Bendlx-Mlsslles  Reliability  ilro.p  before  testing  started.  Test  results 
were  sent  to  Reliability  at  Bendlx-Mlsslles  for  Information  and  follow-up. 

When  a  failure  or  failures  occurred  which  had  a  definite  effect  on  production  and 
the  test  program,  the  Reliability  Group  was  In. mediately  notified  and  suitable 
action  was  taken  to  alleviate  the  problem.  On  such  occasions,  the  subcontractor 
was  asked  to  take  the  necessary  steps  In  bis  plant  and  also  to  contact  his  vendor 
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to  correct  the  existing  situation.  At  the  same  time,  other  subcontractoi s  using 
the  same  part  and  same  manufacturer  were  notified  and  requested  to  check  their 

stock. 


The  ''Standard"  program  modified  the  basic  rules  of  the  "Intermediate*  program 
In  that  the  majority  of  testing  and  preparation  of  detailed  paper  work  was  made 
the  responsibility  of  the  subcontractors.  The  "Standard  Quality  Requirements 
for  Purchased  Material,"  the  basic  quality  spec  for  all  subcontracted  material, 
requires  each  subcontractor  to  establish  and  operate  a  piece  parts  QAT  program 
as  part  of  his  normal  operations  on  TALOS  contracts.  The  responsibility  for 
planning  and  documenting  the  test  program  under  this  program  rests  with  the 
subcontractor.  To  maintain  control,  this  specification  requires  each  bidder  on 
a  subcontract  to  Include  a  description  of  his  proposed  QAT  program  with  his 
proposal.  In  this  proposed  QAT  program  he  describes  his  approach,  concept 
capability  equipment  requirements,  and  program  cost  estimate  for  review  and 
approval  by  Bendix-Mlsslles.  After  a  contract  Is  awarded,  the  subcontractor 
has  a  definite  deadline,  before  delivery  or  the  Hrst  unit,  to  start  testing.  De- 
tailed  test  plans  and  procedures  are  prepared  hy  the  subcontractor  and  approved 
by  Bendix-Mlsslles  Reliability  before  this  testing  can  Btart.  The  responsibility 
for  taking  action,  both  internally  and  with  parts  vendors,  rests  with  the  subcon¬ 
tractor  and  is  carefully  monitored  by  Reliability.  Various  lest  reports  and 
status  reports  are  required  for  control. 

A  considerable  amount  of  paper  work  was  required  to  start  the  program  and  even 
more  Is  required  to  keep  It  going,  but  with  the  funds  available  am,  ne  limit  In¬ 
volved  to  get  the  program  Into  operation,  it  was  thought  necessary  to  use  the 
subcontractors’  existing  test  labor*'  .rles  rather  than  build  a  new  lab  or  expand 
our  own.  This  program,  however,  as  been  effective,  and  will  'ncrease  In  e  - 
fectlveness  with  time  and  Increasec  •>  .-lence.  In  the  ’^ng  r  o,  however,  this 
method  Is  probably  the  most  expensive  and  not  the  most  effec  .  o  for  a  piece  part 
QAT  program.  Even  though  the  over-all  missile  program  may  require  several 
thousands  of  parts,  when  this  amount  is  divided  up  among  eight  or  .me  subcon¬ 
tractors  over  a  period  of  one  year,  the  amount  of  parts  coming  In  per  mont  a 
any  one  place  Is  small.  The  test  sample  In  most  ca°es  has  to  be  made  up  rom 
several  shipments  in  order  to  get  *  'ample  size  of  any  significance.  By  t  s 
time,  the  program  is  well  underway  and  J  a  problem  does  develop  which  re¬ 
quires  any  sort  of  retrofit  action,  the  expense  and  production  schedules  are 

greatly  affected. 


There  are  two  alternate  solutions  to  a  situation  such  as  this  where  the 
contractor,  and  in  some  instances  the  customer,  requires  an  extensive  QAI 
program  on  piece  parts  and  the  contract  Is  divided  up  Into  many  sections.  The 
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first  method  that  could  be  followed  Is  to  have  the  part  vendor  responsible  for 
Initiating  and  carrying  out  the  QAT  program  on  the  piece  parts.  This  type  of 
program  might  be  Impractical  since  each  prime  contractor  has  a  different 
env'ronmental  requirement  and  to  impose  all  the  requirements  on  any  one  part 
would  probably  be  prohibitive  from  the  cost  standpoint  alone. 

The  second  method  which  I  believe  to  be  the  most  effective  and,  over  a  long 
range  program,  the  most  economical,  is  to  set  up  a  central  InBpectton  and  test¬ 
ing  center,  as  Illustrated  In  Figure  9,  controlled  by  the  prime  contractor  but 
set  up  to  operate  as  a  separate  agency.  This  center  would  be  responsible  for 
Inspecting  and  testing  all  electrical  piece  parts  used  In  the  program,  and  would 
also  have  the  responsibility  for  taking  action  with  the  various  part  vendors.  The 
center  would  serve  somewhat  the  same  function  that  the  Reliability  Group  QAT 
effort  does  at  the  present  time  at  Bendlx-Mlsslles.  This  method  would  ensure 
prompt  action  on  test  results  that  are  derived  from  better  statistical  teBt  sam¬ 
ples. 

Since  this  central  agency  would  be  concerned  only  with  the  quality  of  the  piece 
part,  the  prime  contractor  through  his  failure  reporting  system  could  feed  back 
Information  received  In  his  own  plant  plus  that  Information  received  from  his 
subcontractors  and  from  his  customer  In  the  field.  This  system  could  provide 
close  control  over  the  continuing  quality  of  the  piece  parts  and  could  also  provide 
a  direct  control  by  the  prime  contractor  who  utilizes  his  own  Preferred  Parts 
List,  Such  control  could  be  more  effective  than  controlling  subcontractors'  pur¬ 
chases  by  means  of  this  Preferred  Parts  List. 

This  second  plan,  as  presented  here,  Is  only  general  In  nature.  The  details  of 
how  such  a  plan  would  function  and  all  possible  pitfalls  have  not  been  worked  out. 
I  believe,  however,  that  such  a  plan  could  be  worked  out  which  would  benefit  both 
the  prime  contractor  with  lower  production  costs  and  the  customer  with  In¬ 
creased  reliability. 

Before  closing,  I  should  like  to  make  some  remarks  concerning  the  test  equip¬ 
ment  used  to  conduct  these  various  programs.  In  many  cases,  a  strong  argu¬ 
ment  is  put  forth  that  basic  engineering  test  programs,  such  as  our  DME  pro¬ 
grams,  should  have  the  majority  oT  funds  allocated  for  equipment  procurement, 
and  the  production  test  programs,  such  as  our  QAT,  should  use  this  same 
equipment  after  the  original  test  program  Is  completed.  This  Is  not  only  a  false 
argument  but  a  dangerous  one.  CXir  experience  has  shown  us  the  necessity  for 
providing  high-grade  test  equipment  for  all  of  the  programs  described  In  this 
paper.  Early  test  programs  will  frequently  wear  out  test  equipment  to  the  point 
where  accuracy  is  degraded  or  extensive  and  costly  maintenance  is  required. 
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Differences  In  unit  design  between  the  original  engineering  model  and  the  produc¬ 
tion  model,  such  as  the  ones  made  as  a  result  of  the  findings  of  the  DME  and  DET 
programs,  may  render  the  original  equipment  Inadequate  unless  extensively 
modified  or  even  "hay-wired"  to  do  the  job.  Although  temporary  hook-ups  and 
arrangements  may  be  permissible  during  early  engineering  tests — and  this  Is 
debatable — the  demands  for  repeated  use  In  production  teats  and  QAT  programs 
make  permanent  Installations  necessary.  Unless  these  equipment  needs  are 
considered  from  the  start,  the  perennial  conflict  between  production  delivery 
costs  and  chedules  on  the  one  hand  and  QAT  requirements  on  the  other  may  re¬ 
sult  In  the  QAT  requirements  coming  off  second  best.  The  Idea  of  using  ’left¬ 
overs"  and  "hund-me -downs"  for  QAT  should  never  be  tolerated. 

A  complete  test  program,  Including  an  extensive  Quality  Assurance  Test  Pro¬ 
gram,  Is  essential  If  the  military  TB  going  to  require  a  high  reliability  figure  In 
a  program  as  complex  as  the  missile  program  Is  today.  People  have  to  be  made 
to  realize  that  such  an  extensive  testing  program  in  the  missile  Industry  Is  es¬ 
sential  In  producing  a  reliable  product.  I  do  not  know  the  situation  at  other  In¬ 
dustries,  but  I  do  know  that  at  Bendlx-Mlsslles  the  test  program  outlined  here  Is 
a  necessary  rather  than  a  luxury  program. 


•Reference  paper  written  by  R.  H.  Hollis,  "Missile  Test  Equipment  Reliability 
Study,"  presented  at  the  Sixth  Joint  Military -Industry  Guided  Missile  Reliability 
Symposium,  February  1960. 
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FIGURE  2  -  ENVIRONMENTAL  TEST  FLOW  CHART 
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FIGURE  9  -  OAT  PROGRAM  USING  CENTRALIZED  TEST 
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EMPLOYMENT  OP  LAUNCH  SITE  TEST  EQUIPMENT 
FOR  MAXIMUM  SYSTEM  RELIABILITY 

William  B.  Thompion 

Technical  Military  Planning  Operation 
General  Electric  Company 


INTRODUCTION 


This  paper  presents  methods  for  resolving  some  of  the  problems 
encountered  by  the  systems  designer  in  the  early  development 
stages  of  a  missile  ground  support  system.  The  emphasis  la  on 
the  employment  of  ground  test  and  control  equipment  at  launch  site. 
The  work  summarized  in  this  paper  was  done  in  connection  with  a 
subsystem  of  the  Atlas,  but  it  should  be  generally  applicable  to 
other  1CBM  and  IRBM  systems  as  well. 

MAINTENANCE  REQUIREMENTS  OF  BALLISTIC  MISSILES 

The  configuration  and  mode  of  employment  of  a  missile  ground 
support  system  will  depend  on  the  maintenance  requirements  of 
the  particular  missile  in  question,  which  in  turn  depend  on  the  opera* 
tional  requirements  of  the  weapon.  The  time  elapsing  between  the 
placing  of  the  missile  on  the  launch  pad  and  its  firing  or  replace¬ 
ment  by  another  missile  may  amount  to  a  matter  of  years.  Even 
in  the  case  of  a  major  subsystem  of  the  weapon,  the  required  period 
for  uninterrupted  on-line  operation  will  usually  be  measured  in 
weeks  or  months. 

Ehiring  this  lengthy  period  the  missile  must  be  maintained  at  such 
a  level  that  it  has  a  satisfactorily  high  probability  of  being  launched 
successfully  after  very  short  warning.  This  goal  must  be  met  over 
a  period  of  years  without  ever  exercising  the  equipment  fully,  and 
over  a  period  of  months  (or  in  some  cases  even  years)  without  re¬ 
moving  the  misBile  from  the  launch  pad  or  disassembling  it. 

Since  it  is  likely  that  any  complex  weapon  will  suffer  decrements  in 
reliability  after  standing  in  an  inert  state  for  long  periods  of  time, 
it  becomes  apparent  that  one  vital  ground  support  function  will  be  to 
provide  for  the  pc.  iodic  testing  and  checkout  at  the  launch  site  of 
those  circuits  and  components  in  the  missile  which  are  subject  to 


deterioration  with  disuse.  In  addition,  the  operational  requirement  of  a 
very  low  probability  of  malfunction  at  the  beginning  of  a  countdown  may 
make  it  advisable  to  have  continuous  monitoring  of  some  functions  which 
are  performed  by  particularly  unreliable  components,  or  are  subject  to 
a  high  rate  of  turn-on  stress  failure. 

"MISSION  PROFILE"  OF  cAUNCH  SITE  GROUND  SUPPORT  EQUIPMENT 

The  mission  profile  of  the  balliatlc  missile  Is  both  simple  and  difficult 
to  accommodate.  While  in  the  maximum  readiness  state,  the  missile 
must  be  instantly  ready  to  enter  the  final  countdown  phase  with  no 
pre-existing  or  incipient  malfunctions  extant.  And,  depending  on  the 
particular  weapon  system  under  consideration,  it  may  be  required  to 
maintain  this  state  of  readiness  for  many  hours  at  a  time.  Future 
missile  systems  will  undoubtedly  require  even  higher  proportions  of 
.vaximum  readiness  times  than  do  present-day,  .perhaps  eventually 
amounting  to  all  the  on-line  time  of  the  missile. 

The  desirability  of  performing  as  many  maintenance  and  monitoring 
functions  as  possible  at  the  launch  site  while  the  missile  is  fully 
assembled  in  alert  status  is  self-evident  from  the  mission  profile 
just  described.  Only  where  the  failure  rate  of  a  particular  functional 
component  is  very  low,  or  when  a  function  cannot  be  checked  in  the 
assembled  state,  or  when  the  amount  of  support  equipment  required 
to  check  a  function  makes  launch  site  testing  impractical  will  a  re¬ 
quired  test  not  be  performed  at  the  pad. 

Evidently,  it  will  be  necessary  for  some  of  the  launch  site  ground 
support  equipment  to  fulfill  the  same  mission  profile  as  the  missile 
itself.  Primarily,  this  will  be  the  equipment  which  performs  some 
essential  function  in  the  countdown.  A  failure  of  this  equipment  is 
equivalent  to  a  failure  in  the  missile  itself  in  its  effect  on  the  mission 
capability,  and  immediately  puts  the  whole  systenq  down.  The  con¬ 
tinuous  monitoring  equipment  (if  any)  will  also  have  the  same  mission 
profile  as  the  missile,  even  though  it  may  not  enter  the  countdown 
procedure  directly.  Other  launch  site  support  equipment,  while 
essential  in  meeting  the  availability  requirements  of  the  weapon,  is 
not  used  in  the  firing  sequence.  The  latter  equipment  will  have  an 
intermittent  profile,  being  required  only  at  intervals. 

We  can  distinguish  between  two  types  of  intermittently  used  'lp- 
ment:  that  which  is  used  at  periodic  intervals  to  test  the  condition 
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of  the  missile  or  to  perform  some  servicing  function;  and  that  which 
is  called  upon  at  random  intervals  to  facilitate  the  repair  or  replace¬ 
ment  of  a  failed  missile  or  component  thereof.  Failure  of  the  first, 
or  non-critical  type  of  intermittently  used  equipment,  is  not  ordinarily 
a  very  serious  matter,  for  minor  deviations  in  checkout  or  routine  ser¬ 
vicing  schedules  are  unlikely  to  significantly  affect  the  probability 
that  the  weapon  is  in  a  state  of  readiness.  In  the  case  of  the  second, 
or  critical  type  of  intermittently  used  equipment,  chargeable  missile 
downtime  is  incurred  if  the  missile  should  fail  while  the  ground  sup¬ 
port  equipment  is  inoperable,  in  which  case  there  would  be  a  delay 
in  repair  or  replacement  of  the  failed  component  of  the  missile. 


It  seems  apparent  that,  other  things  being  equal,  it  is  highly  desirable 
from  a  reliability  standpoint  to  design  as  much  of  the  ground  support 
equipment  as  possible  of  the  intermittent  use  type,  and  to  minimise 
the  number  and  complexity  of  equipments  whose  mission  profile  (and 
therefore  essentiality)  is  the  same  as  that  of  the  missile  itself.  As 
a  consequence,  monitoring  functiohs  should,  where  the  reliability  of 
the  particular  missile  component  or  subassembly  permits,  be  per¬ 
formed  during  periodic  checks  and  excluded  from  the  firing  sequence. 


DETERMINATION  OF  OPERATING  MODES 

As  indicated  in  the  previous  section,  the  mission  profile  of  much  of 
the  launch  site  GSE  is  predetermined  by  the  nature  of  its  functions, 
Much  missile  handling  and  repair  equipment  is  of  the  intermittent 
type,  while  certain  electrical  and  positioning  gear  is  integral  to  the 
firing  sequence  and  is  therefore  automatically  endowed  with  a  con¬ 
tinuous  type  availability  requirement. 


In  the  in-between  area  is  a  great  aeai  of  necessary  test  and  check¬ 
out  equipment.  The  designer  here  has  the  option  of  specifying 
whether  the  particular  test  is  essential  to  completion  of  the  count¬ 
down,  or  whether  it  can  be  assumed  that  the  function  in  question, 
having  checked  out  satisfactorily  at  the  previous  periodic  examina¬ 
tion,  is  stil  OK.  In  addition,  the  systems  designer  must  decide 
whether  a  function  should  be  monitored  continuously  by  the  GSE  or 
only  periodically  to  insure  a  satisfactorily  high  probability  of  suc¬ 
cessful  launch.  The  present  section  attempts  to  develop  a  mathema¬ 
tical  model  which  will  assist  the  systems  designer  In  arriv.ng  at 
these  dec  I sions . 
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At  the  outset  we  recognise  that  the  support  requirements  of  the 
"maximum  readineai"  miaaile  (if  indeed,  there  are  any  other  kind 
of  on-line  mlaailea  in  future  military  systems)  are  the  most  strin¬ 
gent  from  the  standpoint  of  launch  site  ground  equipment  required, 
and  these  needs  will  in  all  probability  govern  the  design  of  this 
support  equipment.  Furthermore,  the  support  equipment  provided 
must  maximise  not  only  the  probability  that  the  missile  is  ready  to 
begin  the  countdown,  but  also  that  it  will  complete  it  successfully. 

If  we  regard  the  weapon  as  being  composed  of  a  series  of  separate 
functional  components  or  subassemblies,  it  becomes  feasible  to 
analyse  each  subassembly  in  turn  to  determine  the  manner  and  fre¬ 
quency  with  which  it  should  be  checked  at  launch  site  so  as  to  maxi¬ 
mise  the  probability  of  the  system  functioning  through  the  countdown 
without  failure.  In  performing  this  analysis,  it  is  desirable  to  dis¬ 
tinguish  between  "primary"  units  (or  subassemblies)  of  the  weapon, 
and  "secondary"  units.  Primary  units  include  all  airborne  sub¬ 
assemblies,  plus  those  ground  support  units  which  perform  some 
essential  function  in  the  countdown  and  consequently  have  the  same 
mission  profile  (call  it  a  "continuous  critical"  profile)  as  the  missile 
proper,  In  general,  all  primary  components  of  the  system  will  need 
to  be  checked  out  from  time  to  time  (either  at  launch  site  or  in  the 
maintenance  area)  to  insure  that  the  system  is  operating  satisfactorily. 

There  are  in  general  four  possible  modes  of  on-line  testing  which 
can  be  assigned  to  any  primary  subassembly  of  the  system: 

Mode  1.  The  subassembly  is  checked  out  periodically,  but 
not  during  the  countdown. 

Mode  2.  The  subassembly  is  checked  out  periodically,  with 
a  mandatory  check  during  the  countdown. 

Mode  3.  The  subassembly  is  monitored  continuously. 

Adoption  of  the  first  check  mode  for  a  subassembly  means  that  the 
status  of  ground  suppo  t  checkout  equipment  will  have  little  effect 
on  the  probability  of  the  subassembly  being  alert  (i.e.  ,  ready  to  start 
the  countdown),  or  of  successfully  completing  the  countdown  once 
started.  Mode  2  implies  reliance  on  test  equipment  during  countdown, 
while  Mode  3  implies  full-time  dependence  on  the  checkout  gear. 
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Of  course  periodic  maintenance  area  testing  as  well  as  launch 
site  checkout  Is  necessary  to  the  maintenance  of  the  system. 

However,  for  the  purpose  of  analyzing  the  alternative  test  modes 
described  on  the  preceding  page,  It  Is  necessary  to  consider  only 
those  tests  which  can  be  performed  at  the  launch  site. 

The  probability  of  a  given  subassembly  being  alert  at  any  given  time 
while  on  line  depends  on: 

1 .  Its  failure  rate  in  the  environment  encountered: 

2.  the  time  elapsed  since  the  last  checkout; 

3.  the  magnitude  of  turn-on  and  other  stresses  experienced 
during  checkout;  and 

4.  the  mean  time  required  to  replace  the  subassembly  (or 
higher  assembly  or  subsystem  of  which  it  is  a  part), 
or  the  mean  time  to  repair  the  subassembly  in  place. 

Repair  time  and  replacement  time  can  be  lumped  together  as  "mean 
time  to  return  to  service,  "  or  mean  turnaround  time.  The  estimate 
of  mean  turnaround  time  should  include  the  possiblity  that  the  repair 
or  replacement  operation  may  have  to  be  repeated  one  or  more  times 
for  a  single  failure  if  the  first  repair  is  unsuccessful,  or  if  the  re¬ 
placement  subassembly  also  turns  out  to  be  faulty. 

Assuming  that  a  given  functional  component  or  subassembly  Is  alert 
when  the  countdown  begins,  the  probability  that  it  will  still  be  operating 
at  launch  is  a  function  of  the  turn-on  stress,  and  the  checkout  and 
countdown  stresses  experienced,  if  any 


We  define: 

. 

= 

the  probability  that  the  circuits  and/or 
functions  checked  at  the  launch  site  are  In 

4 

working  order,  given  operating  mode  1. 

* 

Li 

the  probability  that  the  circuits  and/cr 
functions  checked  during  launch  site  testing 
survive  the  countdown  without  malfunction, 
given  operating  mode  i. 

1  -  uul 


B 


probability  that  the  subassembly  will  survive 
turn-on  atreaa. 


C  =  probability  that  the  subassembly  will  aurvlve 

checkout  atreaa. 

D  a  probability  that  the  aubaaaembly  will  aurvlve 

countdown  atreaa. 


a 


A 

M 


mean  time  to  return  to  aervlce.  or  remove 

and  replace  time,  for  the  subassembly  or _ 

higher  aaaembly  of  which  it  la  a  part. 

launch  aite  checkout  interval  for  the  syb- 
aaaembly. 

random  failure  rate  during  "OFF"  condition 
of  circuita  and/or  functlona  of  the  aubaaaembly 
checked  at  launch  aite. 

random  failure  rate  of  the  aubaaaembly  in  the 
"ON"  condition. 


The  following  aaaumptiona  are  made  in  the  aubaequent  anaiyala: 

1.  Failurea  are  random  in  time,  i.  e.*  independent  of  time. 
Thia  condition  is  met.by  the  exponential  diatribution  of 
time-to-failure,  e'  A*,  where  Al*  *be  failure  rate  and 

t  the  time. 

2.  The  countdown  may  occur  at  any  ranaom  point  in  time  with 
equal  probability. 

3.  The  time  required  to  perform  a  periodic  launch-aite 
checkout  ia  neg.igible  in  comparison  with  the  vime  be¬ 
tween  checkouts. 

Mode  1  --  Periodic  Checkout  Only 

Let  Aj  symoolize  the  probability  that  a  given  functional  sub¬ 
assembly  of  the  missile  is  alert  (as  regards  those  functions  which  can 
be  checked  at  launch  site)  at  any  random  point  in  time  when  operated 
in  Mode  1;  and  let  Li  be  the  probability  that  the  subassembly  is  not 
only  alert  at  the  beginning  of  countdown,  but  will  finish  the  countdown 
without  a  failure  of  any  of  the  functions  checked  at  launch  site. 


1  »  it02 


There  are  apparently  two  kinds  of  cycles,  each  of  a  different 
length,  which  may  be  encounter  ed  by  the  missile  subassembly  under 
analysis.  Cycle  Type  1,  of  length  s  {the  Interval  between  checkouts) 
is  that  cycle  In  which  no  failures  occur.  Cycle  Type  II,  of  length 
(•  +  r)  occurs  when  a  malfunction  in  the  system  is  detected  during 
checkout  and  includes  the  time  r  necessary  to  get  the  missile  back 
on  the  air.  Figure  2  following,  illustrates  the  situation  graphically. 


•  r 


(CWCKOUT  INTttVAl)  (?U*N  AKOUND  TIMt) 

. . . . ILIglV- 

Flgu#  2.  Probobllity  of  Alert  Statu*  in  Follurs  and  No-Follure  Checkout  Cycle* 

The  probabllityA  that  the  subsystem,  having  checked  out  satis¬ 
factorily  at  the  beginning  of  the  cycle,  is  still  on  alert  status  d^  ^ 
dints  with  time  in  accordance  with  the  exponential  function  e 
At  the  end  of  the  checkout  interval  it  receives  a  stress  from  the 
checkout  procedure  itself,  which  further  (and  abruptly)  diminishes 
the  probability  of  its  being  operable.  At  this  point  the  checkout 
reveals  either  that  the  tests  were  passed  successfully  and  a  new 
cycle  begun  or  that  the  equipment  has  failed  the  checkout.  In  the 
latter  case,  an  interval  of  downtime  r  commences  which  lasts 
until  the  missile  is  "on  the  air"  again. 

It  is  convenient  to  recognise  two  varieties  of  Type  II  (failure) 
cycles: 

1.  That  cycle  where  the  failure  occurs  in  the  waiting  period 
(call  this  Type  HA). 

2.  That  cycle  where  the  failure  is  induced  by  the  checkout 
itself  (call  this  Type  ILB). 
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The  probability  that  the  weapon  is  on  alert  status  at  any  random 
point  in  time  is  given  by: 

(Good  time  In  all  cycles; 

1  (Total  time  in  all  cycles) 


Good  time  in  a 
given  cycle 


From  which: 

I  1 

(Good  time  in 
all  cycles 


Length  of 
cycle 


Fraction  of 
time  good 


ZJ Length  ofl  J  Fraction  of\  ] 
I  cycle  I  '  l  time  good  r* 
All  cycles  '  *  I  i 


Fraction  ofl  / Probability 
time  good  r*l  that  cycle 
f  \  occurs 


while : 


Total  ;ime  in 
all  cycles 


All  cycles 


Length  of 
cycle 


Probability 
of  occurrence 


The  table  below  translates  this  logic  into  the  symbology  of  the 
problem  at  hand. 


TYPE  CYCLE 


(No  failure! 


LENGTH 
OF  CYCLE 


HA  (Failed  in  waiting  period 
IIB  (Failed  at  checkout) 


PROBABILITY 
OF  OCCURRENCE 


,,  -A*. 

(1-e  ) 


( 1-BC) 


The  parameter  v  is  the  mean  amount  of  good  time  expected  when  a 
failure  occurs  during  the  checkout  interval.  This  is  not  exactly  *  . 
(although  —  is  sometimes  a  r<  asonable  estimate)  because  under 
the  exponential  distribution  the  probability  of  failure  is  greater  at 
the  beginning  of  the  interval  than  toward  the  end.  Instead, 


The  probability  (A^)  that  all  the  circuits  and/or  functions  in  a  sub¬ 
assembly  checked  at  the  launch  site  are  on  alert  status  is  computed 
in  accordance  with  the  rationale  set  out  as:* 

,  -A» 

A  =  — ! — - 5—  .  <i) 

1  As  +A  ?  (l-BCe'A  ®) 


♦  See  Appendix  I  forcompletc  derivation. 
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The  probability  that  the  aubaeeembly  operating  In  Mode  1  eurvlvee 
the  countdown  itself  la  a  function  of  the  turn-on  atreaa,  and  aleo  the 
countdown  atreaa  If  thla  la  a  factor.  The  over-all  probability  of  a 
aucceaeful  launch  under  Mode  1  teating  la  given  by: 

L!  *  BD  (2) 

Mode  2  -  -  Periodic  Checkout,  Plua  Check  During  Countdown 

The  probability  of  a  aubaeeembly  being  alert  at  countdown  ia  exactly 
the  aame  under  Mode  2  operation  aa  under  Mode  1.  In  addition. 
howeveV,  it  ia  &lao  neceaaary  for  the  checkout  equipment  itaelf  to 
be  alert.  The  probability  that  a  given  functional  componenet  of  a 
aubaaaembly  and  the  teat  equipment  neceaaary  to  check  it  are  both 
alert  la: 

A2  =  AjA'j  (3) 

A  prime  ia  appended  to  variable  and  parametera  which  refer  to  the 
checkout  equipment,  ao  that  A'^  ia  equal  to  the  probability  that  the 
checkout  gear  aaaociated  with  the  primary  unit  or  aubaaaembly  under 
analyaia  will  be  on  the  alert  at  countdown. 

It  ia  reaaonable  to  aaaume  that  when  a  checkout  of  an  airborne  aub¬ 
aaaembly  or  other  primary  unit  la  performed,  this  check  will  alao 
aerve  to  teat  the  checkout  equipment.  If  thia  la  the  caae,  the  check¬ 
out  interval  a  will  be  the  aame  aa  for  the  primary  unit  being  checked. 
Computation  of  the  probability  of  the  launch  elte  checkout  equipment 
being  alert  ia  then  analogoua  to  the  computation  for  the  primary  unit 
itaelf,  and  is  given  by: 


“  A'a  +  \'  r'(l  -  B'C'  e"  A  ) 

In  thia  caae,  r '  is  the  mean  time  required  for  the  aucceaeful  re¬ 
placement  or  repair  in  place  of  the  checkout  equipment,  A'  l*  the 
failure  rate  of  the  checkout  equipment,  and  B'C*  ia  the  probability 
that  the  checkout  gear  will  survive  the  turn-on  and  checkout  etrea.  ea. 

The  probability  of  successful  launch  uaing  Mode  2  teating  is  given  by: 


(Aj  BCD)  (A'2  B'C'  D') 
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Mode  5  -  -  Continuous  Monitor  in 


Under  the  continuous  monitoring  mode  there  is  no  doubt  about  the 
condition  of  the  circuits  being  monitored.  The  probability  of  alert 
(A4)  for  a  particular  subassembly  is  then  equal  to  the  total  time 
c-.*pad  (h),  minus  the  downtime  experienced  due  to  the  primary  unit 
itself  (d)  and  its  associated  checkout  equipment  (d1).  * 

(6) 


where 

^  »  the  failure  rate  of  the  primary  unit  in  the  "ON" 

condition. 

^  -  the  failure  rate  of  the  monitoring  equipment  (for 

the  primary  unit)  in  the  "ON"  condition. 


r  =  the  mean  replacement  or  repair  time  of  the  sub* 

assembly  or  higher  assembly  of  which  it  is  a  part. 

r'  =  the  mean  replacement  or  repair  time  of  the 
associated  monitoring  equipment. 


Unfortunately,  it  is  nearly  impossible  to  assign  a  dollar  penalty 
to  firing  delays  in  the  combat  situation.  Delay  in  commencing  a 
countdown  because  of  an  inoperative  monitoring  unit  might  result 
in  the  destruction  of  the  missile  on  the  pad,  or  the  loss  of  a  major 
city,  or  in  no  measurable  disadvantage  at  all.  Similarly,  the 
launching  of  a  missile  containing  a  malfunction  might  result  in  the 
survival  of  a  critical  target  which  otherwise  would'have  been 
destroyed.  Or.  if  there  were  no  time  to  repair  the  missile  anyway, 
the  matter  of  whether  or  not  it  were  launched  would  be  of  no  con¬ 
sequence.  Further  rea  :arch  into  this  problem  might  result  in  a 
practical  method  for  evaluating  these  possibilities,  but  for  the 
present  the  problem  seems  insurmountable. 


*See  Appendix  II  for  derivation 
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Being  unable  to  quantify  the  ideal  optimizing  criterion,  the  researcher 
is  forced  to  seek  compromises.  One  possibility  is  to  determine  the 
probability  of  successful  launch  after  x  hours  delay  in  the  countdown, 
for  a  number  of  values  of  x.  Equations  (2),  (5),  and  (7)  give  these 
probabilities  for  (x  =  0).  The  systems  designer  will  then  be  in  a  position 
to  evaluate  (or  to  have  evaluated  for  him)  each  mode  of  operation  for 
several  values  of  x  weighting  each  subjectively  according  to  the  opera¬ 
tional  specifications  of  the  weapon  system  being  designed. 

The  equations  specifying  the  probability  of  successful  launch  with 
x  hours  delay  are  not  difficult  to  write.  In  addition  to  the  parameters 
already  introduced  they  depend  on: 

1,  The  number  of  malfunctions  occurring  during  the  countdown;  and 

2.  The  probability  that  all  malfunctions  can  be  corrected  In  leaa 
than  x  hours. 

Unfortunately  (or  perhaps  not)  considerations  of  brevity  prohibit  the  re¬ 
production  of  these  equations  in  this  paper. 


OPTIMUM  LAUNCH  SITE  CHECKOUT  INTERVAL 

Before  the  relative  merits  of  the  fout-  different  test  modes  for  a  given 
primary  component  of  the  system  can  be  satisfactorily  evaluated,  the 
optimum  launch  site  test  period  s  must  be  determined  for  those  modes 
calling  for  periodic  checkout. 

The  purpose  of  periodic  checks  at  the  launch  site  is  to  make  sure  the 
missile  is  in  alert  status,  that  is,  ready  to  begin  countdown  if  called  upon 
to  do  so.  The  probability  of  a  given  missile  subassembly  being  on  alert 
was  given  as  (Mode  1  operation): 

,  -  X  s 

a  1  -  e 

a  i  =  - — - —  •  u) 

X  b  4Ar  (1  BCe  ) 

The  optimum  value  of  s  can  be  obtained  by  employing  a  slightly 
different  rationale  to  obtain  a  simpler  expression,  which  can  be 
differentiated  ^nd  set  equal  to  zero  to  obtain  the  value  of  s  which 
maximizes  Aj  .  This  approximation  for  the  optimum  value  of  s  is: 


~  /  2r(l  -BC) 


*  Solution  suggested  by  Alan  S.  Manne,  See  Appendix  III. 
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The  approximation  holds  with  remarkable  accuracy  for  the  usual 
ranges  of  X,  r,  B  and  C. 

For  Mode  2  the  approximation  of  Equation  (8)  is  less  accurate 
because  it  fails  to  take  into  account  the  necessity  for  the  check¬ 
out  equipment  to  be  on  that  alert  also.  However,  if  A  (the  probability 
of  the  missile  subassembly  being  alert)  is  substantially  smaller  than 
A'.,  (the  proability  of  the  checkout  equipment  being  alert)  and  A'  is 
fairly  close  to  1.0,  Equation  (8)  will  give  an  acceptably  accurate^ 
approximation  of  the  optimum  checkout  interval  3  for  Mode  2 
operation  also. 

Figure  3  following  shows  the  optimum  launch  site  checkout  interval 
as  a  function  of  the  three  basic  parameters;  mean  time  to  return 
to  service  (or  turnaround  time),  stress,  and  failure  rate.  As 
might  be  expected,  the  lower  the  failure  rate  the  greater  the  optimum 
Interval  between  checkouts.  Reducing  the  failure  rate  per  hour 
to  one-tenth  its  former  value  will  result  in  an  increase  in  the 
optimum  checkout  interval  of  slightly  more  than  three  times  if  the 
other  factors  are  held  constant. 

Also  as  expected,  the  shorter  the  turnaround  time,  the  shorter  the 
checkout  interval  --in  fact,  the  optimum  checkout  interval  is  exactly 
as  sensitive  to  changes  in  turnaround  time  as  to  changes  in  failure 
rate,  but  in  the  opposite  direction. 

The  length  of  the  optimum  checkout  interval  is  directly  proportional 

to  the  probability  of  not  surviving  the  turn-on  and  checkout  stress, 

or  (1-BC).  The  greater  the  turn-on  and  checkout  stress,  the  lees 

attractive  checkouts  become  and  the  greater  the  optimum  checkout 

interval.  The  sensitivity  remains  the  same  as  for  turnaround  time, 

except  that  the  values  which  the  probability  (1-BC)  is  permitted  to  ^ 

take  on  are  limited  to  the  range  0.0  to  1.0.  n 
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ASSUMPTIONS  AND  LIMITATIONS  OF  THE  MODEL 


A  number  of  assumptions  and  limitations  of  importance  which  are 
in'.plicit  in  the  model  should  be  emphasized  at  this  point: 

1.  In-flight  failures  and  any  relationship  which  might  exist  be¬ 
tween  them  and  checkout  frequency  have  been  ignored.  Ordin¬ 
arily  it  may  be  assumed  that  no  such  relationship  exists  ,  and 
where  it  does  exist  is  impossible  to  document  statistically. 

2.  A  lOu  per  cent  checkout  efficiency  has  been  assumed,  which 
is  to  say  that  we  have  considered  the  testing  procedure  to  be 
perfect  in  revealing  any  existing  flaws  in  either  the  missile  or 
the  checkout  equipment  itself.  This  assumption  is  probably 
close  enought  to  the  truth  to  avoid  serious  bias  in  the  case 

of  circuits  and  functions  actually  tested  at  the  launch  site,  which 
is  the  model's  area  of  relevancy. 

3.  The  problem  of  equipment  wearout  has  been  ignored  in 
assuming  an  exponential  distribution  of  failure  times.  In 
general,  the  more  complex  the  equipment  and  the  more  oftnn 
it  is  necessary  to  repair  it,  the  more  likely  it  becomes  that 
an  admixture  of  old  and  new  parts  will  result  in  a  random 
(exponential)  failure  pattern  over  the  entire  life  of  the  equip¬ 
ment.  On  the  other  hand,  comparatively  simple  mechanical 
or  electromechanical  assemblies  (particularly  when  subjected 
to  intensive  use)  may  exhibit  a  wearout  pattern  of  failures 
fairly  early  in  the  program.  The  designer  will  have  to  make  a 
subject  evaluation  of  the  probable  danger  of  wearing  the  equip¬ 
ment  out  prematurely  if  he  chooses  the  continuous  monitoring 
mode  or  if  he  plans  on  very  frequent  checkouts. 

4.  The  model  has  ignored  the  cost  variations  between  different 
operating  modes.  For  instance,  it  would  probably  be  slightly 
more  expensive  to  provide  equipment  for  an  optional  countdown 
checkout  than  for  a  mandatory  check,  etc.  These  cost  varia¬ 
tions  would  probably  be  small  compared  to  the  total  cost  of 

the  system,  however. 

5.  The  model  assumes  that  a  launch-site  checkout  can  be 
interrupted  at  any  moment  to  begin  a  countdown.  This  as¬ 
sumption  is  consistent  with  the  goal  for  checkout  systems 
pertaining  to  all  missile  components. 
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DERIVATION  OF  THE  PROBABILITY  OF 
ALERT  STATUS  FOR  MODE  1  TESTING 


P- 


The  probability  of  alert  statu*  obtaining  in  Mode  1  operation  (A^)  la 
equal  to  the  amount  of  good  time  summed  over  all  cycles  divided 
by  the  total  time  in  all  cycles.  We  define: 

s  =  interval  between  launch  site  checkouts 

r  =  mean  turnaround  time 

\  =  failure  rate  in  the  "OFF"  condition 

BC  =  probability  of  surviving  turn-on  and  checkout 

stresses 

v  =  expected  good  time,  given  a  failure  during  the 

checkout  interval 

We  recognized  three  types  of  cycles:  the  no-failure  cycle,  the 
failure  during  standing  time  cycle,  and  the  failure  during  checkout 
cycle.  Assume  an  exponential  distribution  of  time s-to-failure 
during  the  checkout  interval,  such  that  the  probability  of  surviving 

-  As 

time  a  =  e 


Then 


s(e  *BC)  +  {s  +  r)  v  (l  -  e  }  +  (a  +  f)  s  *  8(1-BC) 


s+r 


s+r 


s(e  JC)  +  (s+r)  (1-e  *)  +  (s+r)  e  (1-BC) 


The  mean  time  to  failure  v  in  a  Type  II  A  cycle  (failure  during 
standing  time)  is  found  by  integrating  between  zero  and  s  the 
derivative  of  the  distribution  function  e  ^  »  multiplied  by  the 

time  variable  and  adjusting  to  make  the  probability  of  failure  be¬ 
tween  zero  and  6=  1.0 


(40) 
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Substituting  this  expression  for 
simplifying: 


in  the  equation  for  Aj  and 


Al 


1  -  e 

At  + 


-  A. 

If  (1  -  BCe 


- 


) 
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DERIVATION  OF  PROBABILITY  OF  ALERT 
STATUS  FOR  MODE  4  TESTING 


The  probability  A  that  the  ayatem  la  In  alert  atatue  In  Mode  4 
operation  la  equal  to  total  good  time  divided  by  total  time,  or 


A4  B 


h  -  d  -  d1 
h 


where; 


Then: 


total  hour  a  on -pad 

downtime  hour  a  due  to  mlaalle 

downtime  hour  a  due  to  monitoring  equipment 

failure  rate  of  mlaalle  in  "ON"  condition 

failure  rate  of  monitoring  equipment  In  "ON" 
condition. 

mean  repair  time  for  mlaalle 

mean  repair  time  for  mlaalle  monitoring 
equipment 


(h-d-d*)  m?)  = 


r'(h-d) 


(h-d-d*)  =  -fTTTj 


Substituting  the  expression  for  d*  given  by  (43)  Into  (42)  gives: 


1  +£U  +^’  ?' 


And  substituting  (44)  in  (43)  gives: 


1  +/ir  +  IL'i' 


1  -  U13 


Substituting  the  values  for  d  and  d'  in  (44)  and  (45)  into  (41) 
the  h  term  cancels  out  leaving: 


A4 


_ 1_ _ 

1  ♦  I1-  T  +  //'  f' 
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APPENDIX  III 

ALTERNATE  MODEL  FOR  DETERMINING 
OPTIMUM  LAUNCH  SITE  CHECKOUT  INTERVAL 


i 


t 


Let  A i  »  the  probability  that  the  primary  unit  ie  "on  the  alert"; 

>  that  ia,  operable. 

i 

BC  ?  the  probability  that  the  j>rimary  unit  survives  turn-on 

and  checkout  etreesea. 


f 


X 


mean  time  to  return  the  ayatem  to  aervice  in  caae  of 
failure  of  the  primary  unit. 

the  time  interval  between  checkout*  (in  hour  a), 
failure  rate  per  hour  of  the  primary  unit  under  analyale. 


Aeaume  that  the  probability  of  no  failure  in  time  a  followa  the 
negative  exponential  distribution,  e  X  For  valuea  of  the  product 
X*  leaa  than  about  0.  25,  the  aeriea  approximation  (1-  X  a)  fore"^  a 
givea  aatiafactory  reaults,  and  thia  aubatltution  ia  employed. 


We  regard  all  cycle*  aa  being  of  equal  length  a.  There  are  three 
types  of  cycles;  no  failure,  failure  before  checkout,  and  failure 
during  checkout.  The  latter  two  type*  require  average  downtime 
for  repair  or  replacement  (or  turnaround  time)  of  T  hour*. 


Since  all  cycles  are  of  equal  length  the  probability  of  alert  status  is 
the  sum  over  each  type  of  cycle  of  the  fraction  of  time  "on  the  air" 
times  the  probability  of  occurrence  of  the  cycle,  or, 


(Probability  of 
alert  status 


I 

All  cycles 


F raction  of 
time  good 


Probability 
of  occurrenc 


This  turns  out  to  be 


A  =  (1  -  X«)  BC  +  (1-Xs)  [\-bC)~  +  X  s 
=  i  .  .  I  (1  -BC)  -  BCf  X 


(46) 
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Differentiating  Equation  (46)  with  reapect  to  e,  we  have: 

dA  «  -A  +  (1  -  BC)  r 
de  2  g2 


Setting  thie  expression  equal  to  aero  yielde  tho  value  of  x  which 
maximises  A: 


RELIABILITY  OF  THE  FUTURE  RADGE  INSTRUMENTATION  COMPLEX  AT  VSMR 

by  Auguat  V.  Rlhactjek  and  Richard  V.  Bullard 
U.rf.  Army' Ordnanca  Corps 
Vhlta  Sands  Mlaalla  Range,  New  Mexico 

In  conaldering  reliability;  It  la  important  that  one  realise  why  it 
la  ao  vital  to  the  operation  of  the  range.  There  are  three  main  reaeona 
why  a  high  reliability  of  a  missile  teat  range  muat  be  established. 

Briefly  these  aret 

(1)  The  coat  of  missile  development.  This  amount  varies;  but  in 
any  event,  it  represents  a  slseable  amount  ranging  up  to  the  order  of  a 
million  dollars  per  test. 

(2)  The  coat  of  operating  the  range.  Specific  figures  of  range 
operation  are  Included  in  the  following  pages,  and  it  will  be  noted  that 
the  sum  la  significant. 

(3)  The  time  invested  in  missile  research.  -In  the  national  neces¬ 
sity  to  keep  pace  in  missile  development,  time  is  one  of  the  most  impor¬ 
tant  medium  spent.  Being  irreplaceable,  suet,  an  investment  swat  be  uti¬ 
lised  to  the  maximum  degree. 

Dp  to  the  year  1958,  the  instrumentation  complex  at  White  8ande 
Missile  Range  was  improved  through  the  development  of  individual  systems. 
If  their  potentialities  warranted,  existing  lnat rumen tat ion  dyatpma  were 
gradually  improved.  Other  systems  with  poor  capabilities  and  potential¬ 
ities  were  abandoned.  When  a  new  system  evolved  that  appeared  suitable 
for  the  range,  it  was  installed  on  a  trial  basis  to  determine  its  capa¬ 
bilities  and  it  was  either  retained  or  removed  after  coeiparlng  it  with 
a  current  standard.  In  this  manner  of  growing,  a  slow  improvement  was 
realised.  To  compare  »-.he  capabilities  of  the  past  with  the  present  is 
difficult  because  standards  of  excellence  have  improved  bacauae  data 
handling  has  changed  and  improved,  smd  because  the  actual  method  of  meas¬ 
urement  and  the  thing  measured  may  not  he  exactly  the  same  now  as  they 
were  originally.  But  considering  all  of  the  variables,  it  is  conserva¬ 
tively  believed  that  our  o/erall  accuracy  is  at  about  3  times  what  it 
was  in  1950.  In  other  words,  in  situations  where  We  were  able  to  measure 
the  velocity  of  a  missile  to  within  plus  or  minus  15  feet  per  second  in 
1950,  we  are  able  to  measure  it  within  plus  or  minus  5  feet  par  second  at 
present.  The  near  future  indicates  tbit  we  vill  soon  need  to  measure  ve¬ 
locity  to  within  plua  or  minus  a  small  part  of  a  foot  per  second.  In 
addition  to  bettar  measurements,  the  capability  of  the  range  to  complete 
more  firings  has  increased  about  25X  a  year.  Last  year  the  range  com¬ 
pleted  approximately  2400  hot  firings.  The  need  for  handling  a  still 
greater  number  of  firings  per  year  is  in  prospect. 


The  grovth  and  Improvement  of  the  range  has  been  accomplished  by 
various  groups  working  to  realize  more  of  the  inherent  capebilitiee  of 
their  assigned  instrumentation  systems.  Tbs  complexity  and  amount  of  the 
instrumentation  systems  have  greatly  Increased  over  the  years  until  the 
point  has  been  reached  where  it  is  necessary  to  have  a  well* integrated 
range  instrumentation  system  to  keep  pace  with  increasing  requirements. 

To  achieve  this  and  a  radically  new  concept  of  a  unified  range  instru¬ 
mentation  complex  has  been  conceived  for  WSMR.  For  thle  concept,  the 
term  AKTBAC  lies  been  coined.  It  is  an  abbreviation  for  "Advanced  Range 
Testing,  Reporting  end  Control."  A  general  outline  plan  for  the  ARTRAC 
System  wea  published  IS  Kerch  19S9  by  the  Integrated  Range  Mlaalon,  WSKR. 

Reliability  vae  a  paramount  consideration  in  the  formulation  of  the 
ARTRAC  plan  Just  as  it  would  have  been  In  devising  any  other  operational 
control  plan.  Reliability  and  dependability  are  extremely  important  at 
White  Sands  Missile  Range  since  the  beslc  reason  for  having  a  missile 
range  is  to  make  measurements  and,  if  they  cannot  be  depended  upon,  the 
range  serves  no  purpose.  The  costs  of  range  time  alao  emphasise  the  need 
for  high  reliability.  Twenty-seven  thousand  dollars  per  hour  la  the  figure 
eat  for  the  cost  of  operating  this  range.  At  this  rate  two  minutes  are 
worth  $900.  One  single  POGO  target  mlsslla  varies  in  coat  from  $3000  to 
$5000  per  round.  Because  of  these  end  other  considerations ,  the  important 
matter  of  reliability  deserves  careful  planning  to  assure  its  achievement. 
This  paper  will  present  the  principles  and  the  planning  that  have  been  used 
In  the  ARTRAC  concept  to  Increase  the  reliability  of  operation  for  the  in¬ 
strumentation  complex  above  that  of  the  individual  instrumentation  systems. 

1.  Reliability  through  the  ARTRAC  structure 

A  missile  range  140  miles  long  and  30  miles  Wide  operated  in  one  sec¬ 
tion  by  e  private  company  and  in  the  other  section  by  civil  service  per¬ 
sonnel  presents  a  formidable  operating  and  control  problem.  One  means  of 
Increasing  reliability  through  use  of  e  control  structure  is  to  make  par¬ 
allel  use  of  several  different  types  of  instrumentation  systems  -  each  of 
which  is  capable  of  providing  the  same  data.  Then,  through  proper  control 
end  coordination  of  all  systems,  range  reliability  will  be  Increased  by 
the  continuous  selection  of  the  best  operational  procedures  end  data  as 
the  flight  progresses.  Operation  in  this  manner  not  only  improves  the  re¬ 
liability  of  measurements  lor  customers,  but  it  slgo  improves  such  lnnsr 
range  functions  as  safety  monitoring  of  missile  flights. 

A  control  structure  can  increase  reliability  of  an  extremely  accu  - 

or  critical  instrumentation  system  by  assuring  that  the  needed  date  for 
support  of  the  system  will  be  available  when  required.  A  control  atructura 
which  provides  for  processed  data  to  be  fed  back  to  tba  ranga  locations 
where  it  is  needed  will  increase  the  reliability  of  tbe  overall  operation. 

A  simple  example  of  one  system  supporting  another  is  that  of  tba  narrow- 
beam  tracking  radar  needing  coarse  date  from  tbe  search  radar  to  find  tbe 
target. 
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Through  proper  coordination  of  all  systems  the  desired  data  can  be 
obtained  from  any  one  operatlon.il  system.  This  unification  approach  was 
chosen  for  AH  TRAC .  ARTRAC  will  establish  a  single  control  structure 
throughout  tha  instrument  complex.  This  structure  will  consist  of  three 
branches:  the  command  control  system,  the  data  system,  and  the  timing 
and  coasnunications  system.  __ 

2.  Command  Control  System 

The  control  structure  for  the  future  range  will  b*  a  continuous 
chain  of  control  positions  with  clearly  defined  control  responsibilities 
extending  from  the  top  rsnge  controller  down  to  the  simplest  control 
points  for  the  operations  of  measurements,  missilery,  targetry,  and  safe¬ 
ty.  The  control  structure  for  ARTRAC  downward  from  the  top  conalata  of 
five  echelons  or  levels.  This  control  chain  will  Insure  prompt  imple¬ 
mentation  of  control  decisions  both  for  normal  operations  and  emergency 
cases  and  thus  will  contribute  to  orderly  progress  of  a  mission.  Control 
of  an  operation  will  be  governed  by  the  entire  set  of  decisions  and  agree¬ 
ments  derived  from  pre-mlsslon  planning  and  scheduling.  The  objectives 
of  control  ere  to  implement  these  decisions  throughout  the  period  Just 
prior  to,  during,  and  Immediately  after  a  missile  flight  and  to  make  any 
decisions  not  anticipated  in  the  pre-mission  planning. 

Under  normal  operating  conditions,  control  responsibilities  will  be  . 
delegated.  Directives  and  Instrumentation  assignments  issued  by  the  rsnge 
control  center  are  to  be  translated  into  system  and  equipment  orders  by 
the  various  operations  at  their  individual  levels.  Decisions  will  be  del¬ 
egated  to  the  lowest  practicable  level.  Reliability  will  be  Increased  by 
avoiding  over-concentration  of  control  decisions  upon  a  single  Individual 
since  responsibility  will  be  delegated  to  an  Intermediate  supervisory 
structure. 

During  an  emergency,  it  Is  realised  that  the  lower  echelons  of  con¬ 
trol  are  not  always  fully  aware  of  the  situation  at  higher  echelons.  As 
a  precaution,  for  such  cases,  ARTRAC  will  give  each  higher  echelon  con¬ 
troller  a  choice  of  over-riding  subordinate  decisions  if  he  believes  it 
to  be  necessary.  This  will  be  accomplished  by  continuously  displaying 
Information  about  th’’  ove  rail  operation  at  the  higher  echelon  of  control 
and  providing  for  direct  channels  to  effect  any  decision.  Thus  reliabil¬ 
ity  of  control  will  be  Increased  by  back-up  through  direct  control  channels. 

Missions  which  reqiire  simultaneous  use  of  all  of  the  facilities  and 
range  area  are  not  the  only  types  tested  at  White  Sands.  The  extent  of 
the  areas  available  pronotes  the  simultaneous  use  of  various  parts  for 
separate  missions.  Controlling  such  multiple  missions  from  a  central 
point  is  not  desirable  when  one  person  is  responsible  for  all  decisions. 
Heavy  stress  would  fall  upon  such  an  individual  and  endanger  the  success 
of  all  missions.  To  avoid  undue  concentration  of  responsibility  for  man¬ 
aging  simultaneous  operations,  under  ARTRAC  mission-control-centers  will 


be  established  in  several  areas  of  the  range.  Each  of  these  centers, 
headed  by  a  controller,  will  be  responsible  for  the  activities  In  its 
Individual  area.  Overall  coordination  will  be  the  task  of  the  chief 
controller,  but  the  complex  multiple  operations  will  be  divided  Into 
several  simpler  operations  wnlch  are  easier  to  control  and  thus  will 
have  higher  reliability. 

3.  Data  System 

Achievement  of  the  use  of  automatic  data  evaluation  and  selection 
procedures  will  be  a  very  important  mile-stone  In  attaining  higher  re¬ 
liability.  Properties  of  good  data  are  reasonably  well  known  for  certain 
systems,  and  data  can  be  evaluated  on  the  basis  of  these  properties.  Aa 
a  simple  example,  If  a  certain  maximum  noise  level  was  specified,  data 
could  be  accepted  or  rejected  from  various  sources  until  the  optimum  was 
selected.  The  procedure  of  using  data  from  one  system  to  Improve  data 
collected  by  ■  second  system  will  also  be  further  exploited.  Such  a  pro¬ 
cedure  results  In  a  more  reliable  output  than  can  be  produced  by  one 
Isolated  system.  Thus  the  strongest  capabilities  of  the  different  systems 
are  combined  to  realise  the  maximum  reliability  available. 

The  ARTRAC  unified  data  system  will  further  increase  the  reliability 
of  the  instrumentation  by  combining  Instrumentation  systems  in  other  ways. 
Some  systems  are  very  reliable  in  operation  but  provide  a  coarser  type  of 
data,  while  some,  which  provide  a  finer  type  of  data,  are  very  susceptible 
to  operational  and  interpretive  errors.  For  the  unified  data  system  of 
the  future  range,  a  very  accurate  system  will  be  backed  by  a  very  reliably 
operating  system,  so  that,  in  case  of  failure  of  the  first,  there  may  be 
assurance  that  at  least  coarse  data  will  be  obtained. 

An  example  Is  found  in  the  resolution  of_the  ambiguities  of  an  other¬ 
wise  highly  accurate  system.  Here  failure  to  resolve  the  ambiguity  would 
cause  failure  of  the  system,  but  the  operation  of  the  resulting  combina¬ 
tion  may  be  made  highly  reliable  through  providing  ambiguity  resolution 
through  the  other  --  even  though  crude  --  system.  As  an  example,  the 
DOVAP  system  is  extremely  accurate  in  measurements  of  velocity  and  posi¬ 
tion.  However,  each  usable  DOVAP  measurement  depends  on  the  history  of 
the  previous  measurements.  Using  DOVAP  only,  an  error  made  in  the  first 
calculations  wovld  be  carried  throughout  the  remaining  computations.  A 
second  instrumentation  system  would  be  of  value  to  determine  that  the 
bias  errors  (resulting  from  an  operational  difficulty  such  as  the  momen¬ 
tary  loss  of  the  DOVAP  signal,  etc.)  would  not  exceed  some  minimum  excur¬ 
sion.  DOVAP  could  then  continue  to  provide  accurate  data  on  position  and 
velocity. 

During  the  gradual  implementation  of  ARTRAC  on  the  range,  the  portion 
of  the  data  reduced  in  real-time  will  be  increased  with  the  ultimate  goal 
of  achieving  most  of  the  data  reduction  in  real-time.  However,  even  where 
real-time  data  is  necessary,  data  et  the  various  points  throughout  the 
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system  will  be  recorded  for  back-up  purposes.  Only  after  the  data  has 
been  obtained  and  the  mission  thoroughly  evaluated  will  these  recordings 
be  erased  or  destroyed. 

The  data  reduction  task  imposes  such  a  heavy  workload  on  the  com¬ 
puting  facility  that  a  single  computer  would  have  to  be  extremely  com¬ 
plex  to  perform  all  data  reduction.  Failure  of  this  central  computer 
would  completely  Interrupt  the  data  flow  during  the  time  the  computer 
was  being  repaired.  For  this  reason,  ARTRAC  data  reduction  will  be  accom¬ 
plished  using  a  group  of  computers,  both  for  real-time  and  post  flight  re¬ 
duction.  These  computers  can  rapidly  be  programed  for  the  various  data 
reduction  procedures  in  use  at  WSMR.  The  computer  to  be  used  normally 
for  post-flight  analysis  will  also  be  capable  of  being  programed  rapidly. 
In  case  of  failure  of  the  computer  engaged  in  real-tUbd  processing,  che 
task  might  be  rapidly  reassigned  to  a  computer  ordinarily  used  for  post¬ 
flight  analysis.  Since  a  delay  of  the  post-flight  analysis  Is  not  criti¬ 
cal,  the  reliability  of  the  overall  computing  facility  will  be  greatly  in¬ 
creased  without  employing  costly  equipment  for  back-up  purposes  only. 

In  the  area  of  data  presentation  equipment,  duplicate  equipment  will 
be  provided.  However,  the  back-up  equipment  for  real-time  data  display 
will  be  kept  to  a  minimum  by  utilising  general  purpose  equipment  that 
may  be  used  for  the  presentation  of  more  than  one  kind  of  data.  One 
single  piece  of  equipment  will  then  serve  as  back-up  equipment  for  a 
series  of  display  devices. 

4.  Communications  and  Timing 

To  provide  flexibility  and  control,  the  communications  system  of 
ARTRAC  will  have  only  one  main  switching  location.  All  communication 
lines  will  converge  from  the  various  areas  to  the  one  point.  Lateral 
communication  will  be  provided  by  switching  at  the  central  point.  Such 
concentration  of  equipment  will  provide  a  less  expensive  means  for  ob¬ 
taining  a  redundancy  of  spare  circuits  and  spare  equipment  for  use  in  the 
event  of  failure  of  the  regular  gear.  Reliability  is  to  be  provided  by 
a  sound  compromise  between  a  system  utilizing  only  working  circuits  and 
a  system  with  1007.  spare  circuits. 

With  emphasis  being  placed  on  automation,  voice-communication  traffic 
will  be  much  less  under  the  proposed  ARTRAC  plan  than  at  present.  The 
circuit  time  required  by  voice  traffic  in  ARTRAC  is  expected  to  be  about 
one-fifth  the  amount  used  at  present.  Status  information  and  countdown 
time  will  be  provided  by  the  automated  portion  of  the  communication  system. 
More  extensive  use  of  teletype  will  be  applied  for  rapid  dissemination  of 
control  information.  The  lightly  used  voice  circuits  will  supplement  the 
status  channels.  In  emergency  cases,  these  voice  channels  may  be  used  for 
maintaining  the  proper  flow  of  information  to  keep  the  range  in  an  opera¬ 
tional  status. 
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Timing  signals  now  utilize  radio  channels  as  wtll  as  wire  line  chan¬ 
nels  and  a  sufficient  number  of  spare  channels  will  be  provided  to  assure 

reliability.  The  timing  signal  itself  will  be  a  pulse  code  which  pennies 

separate  interpretation  of  each  pulse  group  as  a  certain  time  mark  with¬ 

out  relying  on  the  preceding  time  mark.  Any  drop-out  of  the  timing  sig¬ 
nal  will  therefore  have  no  effect  beyond  the  first  mark  received  after 
such  a  drop-out. 


?.  Future  Developments 

The  ARTRAC  plan  is  to  be  the  first  step  in  a  continuing  process  of 
defining,  achieving,  and  refining  an  integrated  system  for  range  instru¬ 
mentation.  Subsequent  revisions  will  make  further  advances  toward  a  more 
efficient  and  unified  system.  Certainly  all  future  revisions  will  be 
made  wlch  proper  consideration  for  reliability.  The  instrumentation  sys¬ 
tems  are  to  be  considered  as  replaceable  blocks;  some  may  be  added  and 
some  phased  out  to  maintain  a  continuously  high  capability.  In  a  unified 
combination,  the  adding  of  a  block  is  to  be  accomplished  through  consid¬ 
eration  of  its  contribution  to  the  whole  complex  plus  its  relationship  to 
any  other  instrumentation. 

Redundancy  does  not  always  strengthen  overall  system  reliability. 
Suppose  telemetry  is  the  main  source  used  to  supply  Information  regarding 
the  pitch  of  a  missile.  If  an  optical  system  were  added  to  also  supply 
information  regarding  pitch,  the  reliability  of  the  overall  instrumenta¬ 
tion  would  surely  be  increased.  •  However,  if  instead  of  the  optical  back¬ 
up,  a  modulated  signal  were  used  to  make  it  possible  for  both  DOVAP  and 
a  second  ITS  system  to  also  supply  pitch  information,  then  (counting 
telemetry)  three  electronic  systems  would  be  supplying  pitch  information; 
but  all  three  systems  might  be  dependent  on  one  stabilised  reference  de¬ 
vice  in  the  missile,  and  all  three  would  be  jependent  on  a  single  missile 
power  supply.  Thus,  the  redundancy  in  this  case  would  probably  not  aid 
the  reliability  of  the  overall  system  a  great  deal. 

The  human  factor  is  also  a  consideration  which  often  adversely  influ¬ 
ences  the  reliability  of  the  range.  It  cannot  be  overlooked  since' a  great 
many  times  the  success  or  failure  of  a  mission  will  depend  on  human  oper¬ 
ators  making  decisions.  The  reducing  of  the  human  influence  can  be  accom¬ 
plished  by  reducing  the  degree  of  responsibility  and  simplifying  the  tasks 
of  the  operating  personnel. 


6.  Conclusions 

We  have  shown  in  the  preceding  discussion  that  equipment  reliability 
is  not  the  sole  factor  in  determining  the  reliability  of  the  range  instru¬ 
mentation  complex.  Individual  instrumentation  systems  can  be  combined  in 
such  a  way  that  the  overall  reliability  of  the  complex  becomes  much  larger 
than  that  of  its  portions.  These  considerations  have  greatly  influenced 
the  ARTRAC  plan  which  provides  for  the  implementation  of  a  truly  advanced 
instrumentation  concept  at  WSMR. 


1  -  122 


ATTENDANCE  LIST 


6TH  JOINT  MILITARY -INDUSTRY  GUIDED  MISSILE  RELIABILITY  SYMPOSIUM 
15-17  FEB  I960,  Fort  Bliss 


Abernathy,  Arthur  H.  -  Holloman  Air  Force  Base 
Acosta  -  Fort  Bliss 

Agee,  William  -  White  Sands  Missile  Range 

Alexander,  Joseph  Eugene  -  The  Martin  Company 

Allen,  I.  L.  (Brig.  Sen.)  -  Dept  of  Defense 

Allen,  James  H.  -  Space  Technology  Laboratories 

Anderson,  George  Peter  -  Remington  Rand  Univac 

Anderson,  Joseph  W,  (Lt.  Col.)  »  Hq.  SAC 

Anderson,  Theodore  L.  -  Autonetics 

Armstrong,  Charles  V.  -  Bureau  of  Naval  Weapons 

Armstrong,  F.  V.  (Wing  Comdr.)  -  British  Joint  Service  Mission 

Arndt,  William  Fred  -  Hughes  Aircraft  Company 

Astrachan,  Max  -  Wright  Patterson  AFB 

Attebery,  Arthur  Hocking  -  Ryan  Aeronautical  Company 

Augenstein,  Bruno  W.  -  Lockheed  Aircraft  Corporation 

Bachiochi,  Ronald  (Sp  U)  -  Fort  Bliss 

Bacon,  Joseph  H.  -  Western  Electric  Company 

Bacon,  Richard  H.  (Mai.)  -  Fort  Bliss 

Baker,  Cyrus  E.  (Capt.)  -  White  Sands  Missile  Range 

Baklini,  Edward  -  Kirtland  Air  Force  Base 

Baldwin,  Robert  D.  -  USAADHRU 

Ball,  Leslie  W.  -  Space  Technology  Lab  Inc. 

Bambenek,  Robert  A.  -  American  Machine  it  Foundry  Corp. 

Barbe,  Martin  -  Space  Technology  Lab.  Inc. 

Barron,  Charles  LeRoy  -  Minneapolis-Honeyvell 
Barton,  Sp-U  -  Fort  Bliss 

Bastedo,  George  R.  -  U.  S.  Naval  Ordnance  Lab. 

Bear,  John  C.  -  TDFW  PWS 

Bechtel,  Lester  leorge  -  The  Martin  Company 
Beck,  Roy  D.  -  U.S.  Naval  Ordnance  Lab. 

Belles,  Hillery  W.  -  Holloman  Air  Force  Bass 

Bellev,  Francis  N.  -  Stavid  Engineering  Inc. 

Belaon,  Phillip  C.  E.  (Lt.  Col.)  -  Dspt  of  Army 

Bender,  Eugene  H.  -  Office  of  Secretary  of  Defense 

Bennett,  Carder  L.  (Capt.)  -  White  Sands  Missile  Range 
Bennett,  Emory  C.  -  Northrop  Corporation 
Bennett,  Theodore  C.  -  U.  S.  Naval  Air  Development  Center 
Benson,  Rodney  G.  -  Western  Electric  Compaiy 
Bentson,  Thomas  G.  -  Western  Electric  Company 
Bergman,  Neil  W.  -  Rocketdyne 
Bernard!,  Mario  M.  -  Raytheon 

Bernstein,  Henry  E.  -  Office  of  Secretary  of  Defense 


1  -  U23 


m 


ATTENDANCE  LIST 

Berry,  Floyd  -  Whit®  Sando  Missile  Rang® 

Betts,  C.  S.  (Op.  Capt.)  -  British  Joint  Service  Mission 
Beyer,  Carlton  -  Office  of  Secretary  of  Defense 
Beyer,  Oeorge  L.  -  Neval  Ordnance  Lab. 

Beyers,  Noman  J .  -  White  Sands  Missile  Range 
Birdvell,  Park  R.  (Capt.)  -  Wright  Patterson  AFB 
Blake,  Thomas  Oaynor  >  5A  Reg.  Industrial  Sec  Bd 
Bloom,  Alan  -  ACF  Industries  Inc. 

Blough,  Richard  Roy  -  University  of  Chicago 
Bluhm,  Richard  W.  -  Bendix  Aviation  Corp. 

Bodson,  Henry  R.  (Lt.  Col.)  -  Fort  Sill 
Borgers,  Hermann  ?.  -  Holloman  Air  Force  Base 

Botkin,  Charles  C.  -  General  Electric  Compary 
Bonrich,  Bsil  J.  -  General -Motors  Corp. 

Boyd,  Julian  Dale  -  Sanders  Associates  Inc. 

Bracha,  Vincent  J.  (MaJ .)  -  ARDC 

Bradford,  F.  Kent  -  Fairchild 

Bradley,  Charles  E.  -  Arlnc  Research  Corp. 

Bradley,  John  R.  -  Douglas  Aircraft  Co. 

Brandos,  Louis  M.  -  Dept  of  Any 
Brans ford,  Horton  (Sgt.)  -  Fort  Bliss 
Braun,  Gerhard  (Dr.)  -  Holloman  Air  Force  Base 

Bredemann,  Richard  V.  -  Emerson  Electric 
Briepohl,  Arthur  M.  -  Atomic  Energy  Conmission 
Brenner,  Claude  W.  -  Allied  Research  Association 
Brierley,  Ralph  E,  -  Bendix  Aviation  Corp. 

Brinhall,  Victor  0.  (Lt.  Col.)  -  Wright -Batters on  AFB 

Brown,  Harold  -  Vought  Aircraft 

Bruce,  Paul  Edward  -  Lockheed 

Brucker,  Wallace  H.  (Brig.  Gen.)  -  Fort  Biles 

Brume,  Ounther  A.  (Maj.)  -  Fort  Bliss 

Brundage,  T,  T.  (Lt.  Col.)  -  Rome  Air  Development  Cents 

Brush,  George  E.  Jr.  -  Fort  Monmouth 

Brzezinski,  Charles  J.  -  Office  of  Secretary  of  Defens 

Buckingham,  Stephen  A.  -  The  Johns  Hopkins  University 

Bukovski,  Joe  -  White  Sands  Missile  Range 

Bulfinch,  Alonzo  -  Picatlnny  Arsenal 

Bullard,  Richard  -  White  Sands  Missile  Range 

Bumpus,  Harold  Truman  -  Telecomputing  Corp. 

Burbank,  Farnura  M.  -  Rocketdyne 
Bussey,  Robert  M,  -  Ford  Motor  Compaiy 
Buus,  Melvin  L.  -  U.  S.  Naval  Ordnance  Lab, 

Cahill,  Starr  W.  -  Rycon  Mfg,  Co. 

Cain,  John  I.  E.  -  Dept  of  Air  Force 
Cameron,  E.  E.  -  White  Sands  Missile  Range 


1  -  h2U 


ATTENDANCE  LIST 


Campbell,  Cyril  C.  -  Corrvair 
Campbell,  Jerry  -  White  Sands  Missile  Range 
Campbell,  John  (FA)  -  Canadian  Joint  Staff 
Carroll,  Thoaaa  Jerome  -  ifylvania  Electronic  3ya 
Carwr,  Oliver  T.  -  Radio  Corp.  of  America 
Cary,  Hall  -  Battelle  Memorial  Institute 
Caahell ,  Charles  F.  -  Fort  Belvoir 
Caato,  Carl  Samuel  (Col.)  -  Joint  Chiefs  of  Staff 
Catalano,  Alfred  A.  (Capt.)  -  Kirtland  AFB 
Cavender,  J,  H.  -  White  Sands  Missile  Range 
Cazanjian,  Rudolph  S.  -  Sylvanla  Electronics  Sya. 

Cedo,  James  John  -  General  Precision  Company 
Chapman,  George  A.  (MaJ.)  •  Fort  Bliss 
Cheak,  Donald  L.  -  l).  S.  Naval  Avionics  Fac* 

Christian,  David  B,  -  Bendix  Aviation 
Christian,  Paul  R.  (Capt.)  -  Wright-Patterson  AFB 
Clark,  Trevor  -  Westinghouae  Electric  Corp. 

Cochran,  Kenneth  E.  -  Battelle  Memorial  Institute 
Cole,  Howard  D.  (1/Lt.)  -  Fort  Bliss 

Colley,  John  L.  -  Western  Electric  Co, 

Conger,  Leslie  R.  (Lt.)  -  Redstone  Arsenal 
Conner,  S.  L.  (MaJ.)  -  Fort  Monroe 

Connor,  William  Stokes  (Dr.)  -  Research  Triangle  Institute 

Cook,  B.  E.,  Jr.  (Maj.)  -  Redstone  Arsenal 

Cooper,  David  (Lt.  Col.)  -  OSAADHRU  - 

Coull,  Charles  (MaJ.)  -  Fort  Bliss 

Courtney,  Robert  -  WSMR 

Cox,  Paul  C,  -  WSMR 

Coy,  John  (Dr.)  -  WSMR 

Crabtree,  LeRoy  D.  -  Dept  of  Amy 

Craig,  Byron  Lee  -  Beech  Aircraft  Corp. 

Crosby,  Philip  B.  -  Martin,  Orlando  - 
Cross,  Robert  K.  -  American  Bosch  Arma  Corp. 

Crowley,  John  James  -  Grand  Central  Rocket  Co. 

Cudlitz,  Stephen  -  Massachusetts  Institute  of  Technology 

Culpepper,  G.  A,  -  WSMR 

Cummings,  Donald  J.  -  General  Motors 

Cummings,  Herbert  G.  -  Rocketdyne 

Cummings,  Melvin  A,  -  Dept,  of  Army 

Curtis,  N,  J.  -  Naval  Ordnance  Test  Station,  China  Lake,  California 
Cushing,  Lloyd  Victor  (Maj.)  -  Canadian  Joint  Staff 
Czajkowski,  Norman  -  U.  S.  Nava?  Ordnance  Lab. 

Dale,  Lawrence  P.  -  Naval  Ordnance  Lab. 

Dallas,  Allen  K.  (Lt.  Col)  -  Redstone  Arsenal 
Dal;/,  Thomas  A,  -  Westinghouse  Electric 


a 


a 


A 


«* 

* 

* 

j 

V 


Si 

V 


ATTENDANCE  LIST 


Davenport,  Charles  C,  -  P-ocketdyne 

Davis,  Charles  R.  -  Wright -Patterson  APB 

Day,  Paul  C.  (Lt,  Col.)  -  Dept  of  Army 

Dvn,  Charles  W,  -  Amaric*»-i  Institute  for  Research 

Dee,  Leo  H.  -  General  Electric 

Dennis,  Robert  J,  -  Cook  Electric  Co* 

DeHita,  Joseph  (Col.)  -  Fort  Bliss 
Des  Countinho,  John  -  Orumann  Aircraft  Engineering 
DeWitt,  Ri chart  H.  -  Office  of  Secretary  of  Defense 
Dhom,  Fred  -  WSMR 

Dicy,  William  W.,  Jr.  (Maj.  Gen.)  -  Office  of  Secretary  of  Amy 
Dickson,  William  J.  -  Western  Electric  Co. 

Dobson,  K.  B.  (Col.)  -  WSMR 

Dodgen,  James  E.  •  D.S.  Naval  Propellant  Plant 

Dorr,  Oeorge  W.  -  Central  Intelligence  Agency 

Doshay,  Irving  -  Aerojet  General 

Doty,  0.  J.  (Capt.)  -  WSMR 

Dotsy,  Raymond  A.  -  D.  S.  Naval  Ord  District 

Draughon,  J.  H«  -  Redstone  Arsenal 

Edelman,  Julian  -  Perkin-Elmer  Corp. 

Edge,  Donald  K.  *  W9ffi 

Edwards,  Thomas  J.  -  Fort  Monmouth 

Ehrenberg,  Marvin  3.  -  Walter  Dorvin  Teague  Assn. 

Eichorn,  William  H.,  II  (Capt.)  -  Ent  AFB 
Elder,  Donald  E,  -  The  Johns  Hopkins  University 
Epley,  Albert  D.  (Lt.  Col.)  -  USCONARC 
Eppele,  Kenneth  R.  -  Wyle  Lab 
Eppenstein,  Herschel  F.  -  Convair 
Ewalt,  Leland  G,  -  aed stone  Arsenal 
$yrrich,  Gerald  Irrr1  n  -  Convair 

Fairbank,  C.  E.  (Maj.)  -  Wright -Patterson  AFB 
Felahaw,  Wendell  L.  - 
Fine,  Aleck  -  Martin  Company 
Finocchi,  Anthony  J.  -  I.T.T.  Corp. 

Firstman,  Sidney  Ir.lng  -  Rand  Corp. 

Fluckey,  Wallace  N.  (Capt.)  -  Fort  Bliss 
Fontana,  Rudolph  E,  -  U.  S.  Naval  Missile  Center 
Ford,  Michel  A.  -  Dept  of  Defense 
Fore,  T.  F,  -  Lockheed  Aircraft  Corp, 

Fosler,  E.  A.  -  Radio  Corp.  of  America 
Foss,  Russ  C.  -  Dept  of  Army 
Foster,  Gayle  H.  (Col.)  -  Fort  Bliss 
Foster,  Levine  W.  -  Oeneral  Electric 
Fox,  Clifton  C.  (Col.)  -  Dept  of  Amy 
Frantik,  Rudolph  0.  -  Atomic  Energy  Commission 

French,  Charles  -  Fort  Bliss 
French,  John  C.  -  NASA  Space  Task  Op. 


1  -  U26 


ATTENDANCE  LIST 


Freund,  Qerald  E.  Western  Electric  Co, 

Frit*,  Harvey  W.  •  Redstone  Arsenal 
Fuller,  Artlrar  L,  (Col.)  -  Fort  Bliss 

□ar&fano  -  Fort  Bliss 

Oartenberg,  Allan  -  Grand  Central  Rocket  Co, 

Gartner,  Gideon  I,  -  Watertown  Arsenal 
Gartner,  Robert  -  WSMR 

Oeluso,  Nicholas  G.  -  Oruman  Aircraft  Engineering 
Gephart,  Landis  S.  (Dr.)  -  Office  Secretary  of  Defense 
Oillr  Collis  P.  -  IBM 

Ginter,  Roll  D,  -  Bureau  of  Naval  Weapons 
Gladstone,  Samuel  R,  -  Raytheon  Mfg.  Co. 

Gleimer,  Bernard  -  Bulova  R  and  D  Labs  Inc. 

Goddard,  Janes  S.  -  Aerojet -General  Corp. 

Golovin,  Nicholas  E.  -  National  Aex*onautics  and  Space  Adn. 
Gonzales,  Charles  P.  -  Philco  Corp, 

Goodrich,  Warren  D.  (LtA)  -  Fort  Bliss 

Gordon,  Harold  J.  -  WSMR 

Oraham,  Eimitt  -  Grand  Central  Rocket  Co. 

Oray,  Lee  Bond  -  North  American  Aviation  Inc, 

Qreen,  Carl  -  WSMR 

Oreen,  Robin  -  Radiation  Inc. 

Gregorito,  Robert  F.  -  Bendi*  Aviation  Corp. 

Oregory,  L.  D,  -  Chance-Vought  Aircraft 
Oriraaila,  Albert  G.  -  Motorola 
Grose,  Vernon  L.  -  Litton  Industries 
Gross,  Thomas  Ralph  -  Federal  Electric  Corp. 

Gross,  Thomas  William  -  -  Lockheed 

Ouelzow,  Clarence  W.  -  Patrick  AFB 

Gust in,  Rita  M.  (Mrs.)  -  Vright-Patterson  AFB 

Ham,  Roland  A.  -  Redstone  Arsenal 

Hanrahan,  Timothy  H.  -  Patrick  AFB 

Harmen,  Raymond  A.  -  U.  S.  Naval  Missile  Center 

Harris,  Robert  G.  (Kaj, )  -  Fort  Bliss 

Harrison,  Louis  H,  (CWO)  -  Fort  Bliss 

Harter,  Wendell  W.  -  Northrop  Aircraft 

Hasty,  Jack  Dudley  Elvood  -  Texas  Instruments  Inc* 

Haverland,  Edgar  M.  -  USAADHRU 

Hawkins,  Robert  A.  -  Avco  R&  Advanced  Dev.  Div. 

Hayes,  Donald  -  Chrysler  Corp. 

Hemstreet,  H.  W.  -  FRM,  WSMR 
Henderson,  George  A.  -  Redstone  Arsenal 
Hendry,  Robert  Needham  -  Douglas  Aircraft  Co. 

Henning,  Richard  Albert  -  Boeing  Airplane  Co. 

Henry,  George  E.  -  General  Electric 
Herbert,  M.  J.  (Capt.)  -  Redstone  Arsenal 


1  -  L27 


ATTENDANCE  LIST 


Herren,  John  C.  (Lt.  Col.)  -  Fort  Bliss 

Hershey,  John  -  BTL,  Whippany,  NJ 

Hicks,  Charles  H.  -  U.  S.  Naval  Avionics  Fac. 

Hillman,  Phillip  L.  -  General  Precision  Co. 

Hintee,  G.  -  WSMR 

Hixon,  R^rmond  L.  -  Marquardt  Corp. 

Hock,  Catherine  D.  -  Office  Secretary  of  Defense 
Hockenbury,  Kelvin  C,  -  Picatinny  Arsenal 
Holliday,  Ivar  M.  -  Space  Technology  Lab.  Inc. 

Hollis,  Raymond  H.  -  Bendix  Aviation 
Holmquist,  C.  0.  (Coradr.)  -  Dept  of  Navy 
Holt,  James  E,  -  Redstone  Arsenal 
Horberg,  Arthur  -  Armour  Research  Foundation 
Horn,  Edgar  T.  -  Rock  Island  Arsenal 
Howell,  William  S.  -  Sund strand  Aviation 
Hronlk,  Richard  H.  -  Mslpar  Inc. 

Hudgins,  Jethro  T.  (Lt.  Col)  -  Fort  Bliss 
Huff,  Thomas  H,  -  WSMR 
Hume,  David  Reid  -  Minneapolis-Honeywell 
Hutchinson,  C.  B.  (Capt.)  -  Fort  Bliss 

Israel,  Edmund  M.  -  AFBKD-ARDC 

Jacks,  Sara  M.  -  Redstone  Arsenal 

Jackson,  Albert  L.,  Jr.  -  Office  of  Secretary  of  Defense 

Jackson,  Raymond  -  WSMR 

Jackson,  Robert  C.  P.  -  Douglas  Aircraft 

Jackson,  Robert  M.  -  The  Garrett  Corp. 

Jacobs,  P-ichard  M.  -  Sylvania  Electronics  Stys. 

Jarrn,  William  K.  -  Redstone  Arsenal 
Jespersen,  Henry  L.  -  Western  Electric  Co. 

Johnson,  C.  C.  (Sfc.)  -  Fort  Bliss 
Johnson,  Dan  W.  -  Strategic  Air  Coimoand 
Johnson,  Edward  Elmer  -  Texas  Instruments 
Johnson,  Jerome  R.  -  Aberdeen  Proving  Ground 
Johnson,  Nelson  -  WSMR 

Johnson,  WilliAi.  C.  -  Bendix  Aviation  Corp. 

Jokgensen,  W,  -  Office  of  Ordnance  Research 

Kahn,  Elliott  H.  -  The  W.  L.  Kaxson  Corp. 

Karbo,  Donald  J.  -  Clevite  Ordnance 
Katsura,  Tomio  -  WSMR 
Keller,  W,  F.,  Jr.  (Lt.  Cmdr.)  •  WSMR 
Kennedy,  Feter  J.  -  Fort  Monmouth  - 

Kieman,  Joseph  M.,  Jr.  (Capt)  -  Office  of  Secretary  of  Army 
Kiessling,  Ernest  A.  (Col.)  -  Rome  Air  Development  Center 
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Killian,  H.  J.  list  Lt.)  -  Fort  Bliss 

Kira el,  Harry  -  General  Electric 

King,  Otie  F,  -  Bendix  Products 

Kirmaird,  Ralph  D.,  Jr.  (Capt.)  -  Fort  Bliss 

Kinsey,  Robert  Sherwood  -  Electro teck  Corp. 

Kitchell,  Thomas  J.  -  Bureau  of  Naval  Weapons 

Kitchen,  Frank  -  W3MR 

Kleban,  Stanley  Jay  -  Motorola 

Klein,  Irwin  C.  (Sp.  la)  -  USAADHRU 

Klitzner,  Harry  Norman  -  Raytheon  Manufacturing  Co. 

Koenig,  Robert  F.  -  General  Electric 

Krainik,  E.  B,  (Lt.  Col.)  -  0.  S.  Naval  Ordnance  District 
Krohn,  Charles  A.  -  Motorola,  Inc. 

Krueger,  Albert  F.  -  Aut one  tics 

Kunzuick,  Gene  Albert  -  McDonnell  Aircraft  Corp. 

Kussmaul,  William  0.  (Lt.  .Col.)  -  Logistics 
Kuzmin,  Walter  Raymond  -  Packard-Bell  Electronics 

Lacey,  Robert  J.  (MaJ.)  -  Air  Force  Ballistic  Missile  Division 
Lambert,  Blake  W.  -  Holloman  AFB 
Lambert,  John  W.  -  Aire  search  Mfg.  Co. 

Lombrecht,  Melvin  E.  -  Vri ght-Patterson  AFB 
Lang,  Donald  J.  H.  (Haj.)  -  Fort  BUss 
Langlie,  H.  Joseph  -  Aerorrutronic 
Larkin,  James  J.  (Lt.  Col.)  -  Fort  Bliss 
Lavond,  Raymond  Eugene  -  Aerojet-Genei->l  Corp. 

Learned,  Robert  Edward  -  Motorola 
LeVan,  William  I.  -  Bell  Aircraft  Corp, 

Levine,  Robert  H.  -  United  Testing  Laboratories 
Levis,  Sherwin  -  Space  Technology  Lab.,  Inc. 

Liggin,  Edvard  H.  -  Thiokol  Chemical  Corp. 

Lipscomb,  Lafar,  Jr.  (Col.)  -  Fort  Bliss 

Logan,  M.  D;  (Wing  Cnidr. )  -  British  Joint  Service  Mission 

Lombardi,  Gerardo  J.  -  Atomic  Energy  Comnlssion 

Lorette,  Sanford  C.  (CVO)  -  Fort  Bliss 

Lover,  John  A.  -  Hercules  Powder  Co. 

Lowrie,  W.  0,  -  Partin  Co, 

Lundholm,  Jere  L.  -  Arthur  D.  Little,  Inc. 

Lutener,  Oral.  C.  -  Fort  Bliss 

Lyle,  Douglas  A.  -  Western  Hectric  Co, 

MacCarley,  J.  A.  -  Marcuardt  Aircraft  Co. 

MacCiemT^,  George  L.  -  United  Testing  Lab. 

Macho,  fcdward  John  -  North  American  Aviation,  Inc. 

Maertens,  Thomas  F.  (Kaj.)  -  Office  of  secretary  oi  «r>ny 
i^itzen,  uonaid  R.  -  The  Martin  Co, 

Mallon,  Patrick  J.  -  Hughes  Aircraft  Co. 
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Marc  rum,  Robert  H.  (1/Lt.)  -  Fort  Elisa 

M&rko ,  Frank  -  Frankford  Arsenal 
Martin,  C,  C.  (Capt.)  -  Fort  Bliss 
Martin,  Harry  Ward  -  Sperry  Gyroscope  Co. 

Martin,  Justin  E.  (Capt.)  -  Fort  Bliss 
Marzec,  Leonard  Edvard  -  Canadian  Joint  Staff 
Masters on,  Robert  Janes  -  Martin  Company 
Matosoff,  Henry  I.  -  Cannon  Electric  Co. 

Matthews,  Charles  W.  (Lt.  Col.)  -  Fort  Sill 
Mayer,  Matthew  A.  -  General  Motors 
Mayes,  Richard  A.  -  United  Electro dynamics  Inc. 
McCachern,  William  T,  (Lt.  ColO  -  Fort  Bliss 
McCallum,  Hargro  -  Frankford  Arsenal 
McClure,  Julius  loung  -  Uonvair 
M^Oool,  VHVHpm  A.  -  VJWR 
McCormac,  Billy  M.  (Kaj.)  -  USCONARC 
McCracken,  Bruce  0,  -  General  Electric 

McDonald,  Robert  C.  (Col.)  -  Fort  Bliss 
McGinnis,  Frank  -  Sperry  Gyroscope  Co. 

McGlothlin,  William  Herschel  -  Rand  Corp. 

HcKinzie,  Melvin  A.  -  Wright-Patterson  AFB 
McLaughlin,  Charles  E.  -  Wright-Patterson  AFB 
McMaster,  Alexander  C.  -  Convair  _ 

Meikle,  Donald  K.  -  Chrysler  Corp. 

Meister,  David  -  Convair 

Mellnik,  Stephen  M.  (Brig.  Oen.)  -  Fort  Bliss 

Manges,  William  Randall  -  Burroughs  Research  Center 

Mercuric,  Saluatore  P,  -  General  Electric 

Merritt,  Robert  B.  -  General  Electric 

Mika,  Stanley  E.  (M  Sgt.)  -  Fort  Bliss 

Millard,  Guy  A.  Jr.  -  U.  S.  Naval  Ordnance  Test  Station 

Miller,  Thomas  Kenneth  -  Oilfillan  Bros. ,  Inc. 

Miller,  William  F.  -  Navy  Weapons 
Mitchell,  James  W.  -  Frankford  Arsenal 
Hitman,  Henry  D.  (Capt.)  -  ARGMA 

Moehle,  Frederick  L.  W.  -  The  Johns  Hopkins  University 

Monroe,  William  Ransom  -  Convair 

Moore,  Victor  E.,  Jr.  -  Douglas  Aircraft  Co. 

Morris,  Jack  C.  (Maj.)  -  HAFB 

Morrison,  Edward  John  -  Texas  Instruments,  Inc. 

Morrison,  Stephen  C.  -  Space  Technology  Labs.,  Inc. 

Moskovitz,  Abraham  I,  -  Aerojet  General 

Moskovitz,  Arthur  -  North  American  Aviation,  Inc. 

Murray,  Kenneth  V/.  -  General  Electric 

Murray,  William  A.  -  Goodyear  Aircraft  Corp. 

Mutch,  William  Warren  -  Naval  Research  Lab. 
layers,  Robert  Eugene  -  Minneapolis-Honeyvell 
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Hjrlea,  Clyde  Wesley  -  McDonnell  Aire reft  Corp. 

Nagy,  George  -  Goodyear  Aircraft  Corp* 

Neese*  Janes  Arthur  -  McDonnell  Aircraft  Corp* 

Nelson*  R.  W.  -  WSMR 

Neve 11*  Joseph  C.  (Lt.  Col.)  -  Port  Bliss 
Nichols,  Robert  V*  -  General  Electric 
Nielsen*  I.  L.  -  General  Motors  Corp. 

Nietsch*  Herman  X.  -  Robinson  Technical  Products 
Norland,  Raymond  J.  -  Wri ght-Pattarson  APB 
Nottingham,  Ralph  B.  -  North  Electric  Co. 

Oakley*  Paul  D.  -  Diamond  Ordnance  Fuse  Lab. 

Odell*  Patrick  L.  -  Klrtland  APB 

Oelke*  Harlan  J*  -  Diamond  Ordnance  Fuse  Lab 

Gkun*  Abe  M*  -  RCA 

Orkand*  Donald  S.  -  Picatinny  Arsenal 

Orris,  Derrell  Boyd  -  Philco 

Osborn,  Miles  -  WSMR 

Oven,  Norman  B.  -  Port  Bliss 

Pabst*  William  R*  -  Bureau  of  Weapons 
Page,  W.  W.  -  WSMR 

Patterson,  Melrin  A.  -  Bendix  Arlatlon  Corp. 

Paulsen,  Frank  -  Packard -Be 11  Electronics 
Peck,  Donald  S,  -  Bell  Telephone  Lab, 

Pierce,  Chester  -  Air  Research  and  Dev.  Com. 

Pendry,  R.  A.  (Sq.  Ldr.)  -  British  Joint  Service  Mission 

Penfold,  Lyle  Elmer  -  Consolidated  Western  Steel 

Peratlno,  George  S.  -  Bureau  of  Naval  Weapons 
Peterson,  Kendall  (MaJ.)  -  Port  Bliss 
Peterson,  N.  Marcus  -  Sperry  Utah  Engineering  Lab. 
Phelps,  Robert  Chapman  -  General  Dynamics 
Phillipps,  Elmer  Albert,  Jr.  -  Whittaker  Gyro 
Polking,  Urban  H.  -  WSMR 

Pollock,  atan^ey  I.-  U.  S.  Naval  Ordnance  Lab. 

Porter,  Donalu  K.  -  Fort  Bliss 

Powell,  Harry  R.  -  Space  Technology  Labs.,  Inc. 

Powell,  Roger  D,  (Capt.)  -  USARADoCH 
Power,  G,  W.  (Brig.  Gen.)  -  Fort  Bliss 
Powers,  Aaron  B.  -  U.  S.  Naval  Ordnance  Lab. 

Procassinl,  Andrew  Anthony  -  Motorola 
Prcvencio,  Paul  -  WSMR 

Raines,  Martin  L.  (Lt.  Col.)  -  NASA  Space  Task  Group 
Randall,  Thomas  R.  (Capt.)  -  Port  Bliss 
Rasmussen,  M.  I.  -  WSMR 
Ray,  Jack  Burton  -  Convair 
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R*7,  John  -  Bureau  of  Weapon* 

Retd,  John  Henry  Fredrick  -  Canadian  Joint  Staff 

Reeve,  Edward  A,  -  IBM 

Relnloke,  Herbert  0.  -  North  Electric  Co, 

Rice,  H,  R,  (MaJ . )  -  Fort  Monroe 

Rice,  John  A.  -  Raytheon  Kfg.  Co, 

Richardson,  Jane  a  D.  -  Electro-Mechanical  Rea.,  Inc. 
Riedaech,  Nelson  -  Boeing  Airplane  Co, 

Risk,  bon&lo  Leslie  •  Convair 
Rivera,  James  B.  -  RCA 

Roberts,  Arnold  N.  P,  (Maj.)  -  Dept  of  Any 
Roberts,  Donald  B.  (Col.)  -  Dept  of  Air  Force 
Roberts,  Ernst  E.  (Capt.)  -  Fort  Bliss 
Roberta,  Ray  0.  (Lt.  Col)  -  HFAB 
Roberts,  William  M.  -  Aberdeen  Proving’  Ground 
Rcdkfcr,  Frsncis  J.  (Lt.  Col.)  -  Fort  Bliss 
Rogers,  James  P,  -  USAADHRU 

Roll  in,  Russell  A.  Jr.  -  University  of  Michigan 

Rendg,  Harry  0.  -  Hoffaan  Labs.  Div.  - 

Rooter,  Albert  (Capt.)  -  Norton  AFB 

Roper,  Kenneth  (Capt.)  -  AF  Ballistic  Missile  Agency 

Rowe,  Geoffrey  C«  -  Canadian  Joint  Staff 

Rumin,  Clifford  H.  «  Bureau  of  Naval  Weapons 

Rumer,  William  Isaac  -  Rand  Corp. 

Rupe,  Jesse  C.  -  USAADHRU 

Rush,  Robert  I,  (Maj.)  -  Fort  Bliss 

Russell,  Charles  Woodrow  -  Republic  Aviation  Corp. 

Russell,  James  Clifford  .  -  Raytheon 

Russell,  Sam  C,  (Maj.  Oen.)  -  Fort  Bliss 

Rutter,  George  W.  (Maj.)  -  Wright-Patterson  AFB 

Safford,  Edvard  J.,  Jr,  -  Fort  Bliss 
Sager,  Wesley  C.  (Lt.  Col.)  -  Fort  Bliss 
Schulze,  Henrich  A.  -  Redstone  Arsenal 
Schmidt,  Harry  F.  -  NOTS,  China  Lake 
Schmidt,  Robert  Eugene  -  Raytheon  Kfg.  Co. 

Schuldenf rei ,  Man. in  -  Bureau  of  Naval  Weapons 

Schwinge,  Heinz  T.  -  HAFB 

Se&rle,  Richard  J.  -  Bendijc-Pacific 

Seeger,  Donald  E.  -  Picatinry  Arsenal 

Sewell,  Walter  S,  (Col.)  -  Office  of  Ordnance  Research 

Shanley,  Thomas  J.  B,  (Col.)  -  Dept  of  Arr*y 

Shapero,  Albert  -  Stanford  Research  Inst, 

Shenk,  Donald  H.  -  Redstone  Arsenal 
Sheridan,  Frank  L,  -  Wright-Patterson  AFB 
Sheridan,  John  P.  -  Sprague  Electric  Co. 

Sherburne,  R.  K.  (Dr.)  -  WSMR 

Shonerd,  Clyde  -  WSMR 
Shutt,  Phillip  S.  -  WSMR 
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Simon,  H*ni7  (MaJ.)  -  Dept  of  Air  Fore* 

SJoberg,  John  Calvin  -  Avco  Corp. 

Skavlenski ,  Theodore  L.  (Lt.  Col*)  -  Eglln  AFB 
Smith,  Chandler  C.  (Lt.)  -  Sandla  Base 

a*ith,  C.  D.  (Dr.)  -  The  Mitre  Corp. 

Smith,  Marlon  P.  -  Minneapolls-Honeywell 
Smith,  Richard  K.  -  Bendlx  Products  Co, 

Smith,  Robert  0.  -  USAADHRU 

Snodgrass,  John  W.  -  Martin  Co. 

Snow,  Edward  0*  «  WSHR 

Spangler,  Richard  S.  (Col.)  -  Fort  Bliss 

Spears,  Otis  Sheldon  -  Fort  Sill 

Spencer,  Hilton  T.  (Capt.)  -  Sandla  Base 

Spergel,  Jack  -  Fort  Monmouth 

Stevens,  John  D,  (Col.)  -  Rq.  Air  Defense  Center 

Stallard,  Robert  L,  -  Martin  Co. 

Steele,  Orris  A.  -  HAFB 
Steinberg,  Alvin  -  Redstone  Arsenal 

Steiaber,  Lawrence  H.,  Jr.  -  Battelle  Manorial  Institute 
Steepler,  Samuel  -  The  W.  L.  Maxson  Corp. 

Stengleln,  Thc*aa6  A.  -  h  Corp. 

Stephan,  Martin  H.  -  The  Johns  Hopkins  University 
Stephens,  Fioyd  R.,  Jr.  -  Firestone  Tire  &  Rubber  Co. 
Sternberg,  Irving  Paul  -  Telecomputing  Corp, 

St-even,  Martin  H.  -  John  Hopkins  University,  APL 
Stevens,  John  D.  (Col.)  -  Ho.  Air  Defense  Center 
Stevens,  Martin  -  General  Electric 
Stevens,  Paul  M.  -  Ford  Motor  Co. 

Stewart,  Arlington  Kenneth  -  Raytheon  Mfg.  Co. 

Stollcr,  David  Samuel  -  Rand  Corp, 

Storer,  Robert  L.  -  Rocketdyne 
Strick,  Paul  J.  -  Wright-Patterson  AFB 
Strider,  Thomas  P.  (Capt.)  -  Fort  Bliss 
Strobe 1,  Howard  McClane  -  NI  Naval  Shipyards 
Studdard,  James  L.  -  Redstone  Arsenal 
St.  James,  L.  N.  -  Bell  Telephone  labs. 

Sundt,  William  M.  -  Atomic  Energy  Conmission 

Surber,  J.  W.  (1st  Lt.)  -  ADC  to  DCF 

Sussman,  David  L.  -  Fort  Monmouth 

Swain,  James  W,  (Maj.)  -  Fort  Bliss 

Sweimlsr,  Donald  J.  (Capt..)  -  Ent  Air  Force  Base 

Swift,  Edward  C.  -  Pratt  &  Whitney. Aircraft 

Tanner,  Joseph  J.  (Maj.)  -  Fort  Bliss 
Teresi,  James  Madison  -  Convair 
Thames,  Torben  0,  -  NA  Aviation,  Inc. 

Thorar.o,  Charles  R.  -  Chance  Vought  Aircraft 
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Thomas,  James  W.  -  U.  S.  Naval  Ordnance.  Lab. 
Thompson,  Francis  A,  -  Bureau  of  Weapons 
Thompson,  William  B.  -  General  Electric 
Thorgersen,  Harold  L.  -  USAADHRU 
Thrower,  Martha  W.  -  Redstone  Arsenal 
Townsend,  Alan  R.  -  Allison 
Truelove,  Alan  John  -  General  Electric 

Ulanowicz,  Anil  W.  (MaJ.)  -  Fort  Bliss 
Ullman,  Richard  -  Fori  Instrument  Co. 

Ulrey,  Roy  C.  -  Wright-Patterson  AFB 
Upton,  Albert  P.  -  Minneapolis-Honeywell 

Vanderbeck,  John  P.  -  U.  S.  MOTS,  China  Lake 
Van  Der  Horck,  Max  J.  -  Ramo-Wooldridge 
Varney,  Zenas  Joseph  -  McDonnell  Aircraft  Corp. 
’aughan,  William  H.  (MaJ.)  -  Fort  Bliss 
/erfceck,  William  M.  -  Melpar,  Inc. 

Vernon,  William  Burke  -  Boeing  Airplane  Co. 
Vezeau,  Waldo  A.  -  Emerson  Electric 
Vogel,  v.illian  R.  -  Ordnance  Weapons  Command 
Voosen,  Bernard  John  -  Rand  Corp. 

Wagner,  Robert  L.  -  Picatinrjy  Arsenal 
Walker,  Peter  Gautier  HI  -  Gilfillan  Bros.  T  ic. 
Valters,  Allie  L.  (Capt.)  -  Fort  Bliss 
Watson,  John  V,  Jr.  (Capt.)  -  Fort  Bliss 
Watts,  Norman  W.  -  U.  S,  Naval  Weapons  Lab. 

Webb,  Willis  L.  -  WSMR 

Webster,  Samraie  L.  -  Hughes 

Welch,  PaulW.,  Jr.  -  WSMri 

Welker,  Everett  Linus  -  Arinc  Research  Corp. 

V'endt,  Robert  R.  -  Bendix  Aviation  Corp. 

Wertz,  Clyde  E.  (Lt.  Col.)  -  Fort  Bliss 
West,  Einar  (Dr.)  -  Hercules  Powder  Corp. 
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White,  David  -  Sparry  Utah  Engineering  Lab. 

White,  V.'.  T.  -  Sperry  Oyro 
Whiting,  Sanford  V.  -  Fort  Bliss 
Wiegand,  Walter  J.  -  Hughes  Aircraft  Co. 

Wier,  James  Edward  -  Temco  Aircraft  Corp. 

Wier7'  ckl,  Stanley  T,  -  Watertown  Arsenal 
Williams,  Jerry  R,  (l/Lt.)  -  Fort  Bliss 

Williams,  Ralph  I.  (Lt.  Col.)  -  Fort  Bliss 
Williams,  Robert  T.  -  Arinc  Research  Corp. 
Williams,  Wyman  L.  -  USAADRHU 

Willing,  Walter  R.  (Capt.;  -  Canadian  Joint  Staff 
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Wilson,  Thomas  W.  (MaJ.)  -  Redstone  Arsenal 
Wirth,  E.  -  Ford  Motor  Co. 

Wittinpn,  inward  0.,  Dr.  -  Office  of  Secretary  of  Array 
Wood,  Arthur  *•.  Jr.  -  Leach  Corp. 

Wood,  Ernest  C.  -  Office  of  Secretary  of  Defense 
/  Wray,  Stanley  T.  (MaJ.  Oen.)  -  Vright-Patterson  AFB 

Wright,  Albert  '•%  -  USAAuHRTJ 

Wrobleslci,  William  J.  -  University  of  Michigan 

+  Tetter,  Wlllian  P.  -  North  American  Aviation 

Zabrlskie,  Hollis  C.  -  HAFB 

Zandstra,  Thomas,  Dr.  -  VSHR 

Zlomke,  Clayton  G.  -  Rocketdyne 

Zorger,  Paul  Harvey,  Sr.  -  Ramo-Wooldriage,  Inc. 

Zqicker,  Vernon  M.  -  Allison 

Zucker,  Burton  D.  -  Ford  Instrument  Co. 
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